During its I/O developer conference, Google today announced that it’s working on an experiment called Universal Translator. As its name suggests, this tool aims to translate videos from one language to another, while retaining the overall tone and vibe. This means that not only does the utility translate the audio from one language to another, but it also mimics the voice, tone, and facial expressions of the speaker. That’s right, the outputted video of a person speaking will sync the lips’ movements based on the target language’s utterances.
Considering that this tool can be misused to create problematic deepfakes, Google is limiting access to Universal Translator. Consequently, only authorized partners will get to utilize its power for constructive projects, while users in general will be excluded. While this move limits the promising potential of Universal Translator, it helps keep bad actors at bay.
As the graphic in the photo above reflects, Universal Translator first detects the words and translates them. Then, it checks the tone of the speaker and what they’re stressing on. After combining these two aspects, it generates the speech in the target language. Lastly, it syncs the lips of the speaker in the video based on the pronunciations of the AI-generated voice.
Apart from Universal Translator, Google launched today other AI-powered tools, including improved navigation in Maps, a smarter Google Search, and more. Notably, the company also launched some of its highly anticipated hardware, such as the Pixel 7a, Pixel Fold, and Pixel Tablet. However, considering the current trends revolving around Artificial Intelligence, the company dedicated a significant chunk of its keynote to preview its AI-powered tools, rather than mostly focusing on its hardware and upcoming Android OS features.
Those interested in checking out the main keynote can rewatch the stream after it concludes through Google’s official YouTube channel.