Cyberpunk 2077 Uses AI To Lip Sync In Multiple Languages

Cyberpunk 2077 Uses AI To Lip Sync In Multiple Languages

Cyberpunk 2077 is set in a vast open world populated by hundreds of NPCs. With such a large scope (and the challenges of the pandemic), animating every character in the game by hand is a near-impossible task — but with the help of advanced AI processes, the CD Projekt Red team has found a way to get around the traditional motion capture process.

To build on the sense of realism and quality established in earlier games like The Witcher III, CD Projekt Red is employing JALI, an advanced AI animation tool designed to produce realistic and expressive speech animation with minimal oversight.

Traditionally, this work is completed by actors in motion capture rigs, but the rise of the coronavirus pandemic has meant this process is no longer viable. Instead, JALI will take the reigns and help shape the NPCs of Cyberpunk 2077‘s world.

JALI is a procedural tool that breaks down animation into multiple categories including character model, face shape, muscle movement, speech, speech style, eyes and brows to create more realistic and believable facial patterns during speech. It’s set to be used by every voiced character in the world of Cyberpunk 2077.

It’s also going to be used for the localisation process.

Cyberpunk 2077 is set to have 10 voiced language options including English, French, Japanese, German, Spanish, Polish and Simplified Chinese. JALI is being used to ensure these vocal performances are dubbed accurate and realistic for every language with a focus on correct lip syncing.

Lip syncing is a feature that can really make or break a game’s level of immersion. It might seem inconsequential in the long run, but every detail adds up, particularly in a game world as intriguing as Cyberpunk 2077.

You can hear more about the animation process behind Cyberpunk 2077 in the video below. It’s fairly long, but very interesting and shows just how far animation (and AI) in video games has come over the past decade.

Cyberpunk 2077 is still set for release on November 19. Stay tuned to Kotaku Australia for all the latest news and updates on the game.


The Cheapest NBN 1000 Plans

Looking to bump up your internet connection and save a few bucks? Here are the cheapest plans available.

At Kotaku, we independently select and write about stuff we love and think you'll like too. We have affiliate and advertising partnerships, which means we may collect a share of sales or other compensation from the links on this page. BTW – prices are accurate and items in stock at the time of posting.

Comments


2 responses to “Cyberpunk 2077 Uses AI To Lip Sync In Multiple Languages”

Leave a Reply

Your email address will not be published. Required fields are marked *