No longer will NPCs spew a random line from a pre-written script when interacted with. Nvidia’s Avatar Cloud Engine, or ACE for short, ups the intelligence of our virtual denizens by enabling the ability of lifelike interactions through speech conversation and character animation.
First introduced in 2022, the custom AI model foundry made its return at Computex 2023 in Taipei, and Nvidia’s calling it a “peek at the future of games.” The glimpse came by way of a short scene from a demo called Kairos, developed in collaboration with Convai.
Built in Unreal Engine 5, the impressive tech demo uses the latest ray-tracing features and Epic MetaHuman characters to create a Cyberpunk-esque ramen shop and its non-playable proprietor named Jin which you can see down below.
It happens all too fast, but the gist of it is, instead of a traditional ‘dialogue tree’ interaction we see in many of today’s modern RPGs, through the power of ACE, the player instead simply asks the NPC a question and gets a response in real time. Spooky.
There’s a lot going on in the background. The green team combines Nvidia NeMo, Nvidia Riva and Nvidia Omniverse Audio2Face to deliver its optimised AI foundation model for speech, conversation and character animation.
Breaking it down, NeMo is used for building, customising and deploying language models based on proprietary data. Nvidia Riva, on the other hand, allows for automatic speech and text-to-speech recognition, enabling the live speech conversation we see between Jin and Kai. Finally, Omniverse Audio2Face wraps it all up in a pretty little bow by matching speech audio to create a realistic expressive animation and lip syncing.
“Generative AI has the potential to revolutionize the interactivity players can have with game characters and dramatically increase immersion in games,” said John Spitzer, vice president of developer and performance technology at Nvidia. “Building on our expertise in AI and decades of experience working with game developers, NVIDIA is spearheading the use of generative AI in games.”
Nvidia says developers can easily and cost effectively integrate the entire Nvidia ACE for Games solution or use only the components they need. Can you imagine the possibilities?
