Swedish sci-fi adventure flick Watch the Skies is using a technique called ‘visual dubbing’ to alter the cast’s lip movements and map them to dubbed English dialogue for international audiences – without reshooting a single scene.
That’s thanks to Los Angeles-based movie-making AI firm Flawless, and its TrueSync technology that analyzes the entirety of an actor’s performance, utilizes deep learning to create a volumetric 3D representation of their face, and then adapts their mouth movements in the footage in post-production.
TrueSync has actually been around for a few years, and was notably used to remove F-bombs from the PG-13 edit of the 2022 British survival horror movie Fall (which happened to have been directed by Flawless CEO Scott Mann).
Fall (2022 Movie) Official Trailer – Grace Caroline Currey, Virginia Gardner
Check out the trailer above: you’d swear too if you found yourself at the top of a giant radio tower without a way down. TrueSync helped replace 35 cusses with less offensive phrases like ‘fricking,’ without the need to reshoot a single scene. Here’s what that looks and sounds like.
Flawless use of TrueSync on the movie “Fall”
This time around, Flawless’ tech is being used to visually dub an entire movie. Watch the Skies, which debuted in its home country back in 2022 as UFO Sweden, follows a teenager searching for her missing (and possibly abducted by aliens) father with a UFO watchers’ club. The original actors dubbed their own performances into English at a recording studio, so their Swedish accents are intact.
WATCH THE SKIES – World’s first AI visual dubbed feature film
“We believe that the audience will experience it as an English-language film without noticing the technical transformation,” said Albin Pettersson from the film’s production collective Crazy Pictures. “The result is much better than we ever dared to hope for.”
Flawless
The tech also uses a rights management system to ensure AI-powered editing is carried out only with the artists’ consent. TrueSync is also said to help save up to 50% in costs of redoing scenes for dubbing, compared to reshooting them.
If you’re in the US, you’ll get a chance to see for yourself just how realistic it is over the course of an entire movie. Watch the Skies plays at silver screens across the country starting May 9.
Source: Crazy Pictures