GPU giant Nvidia is getting into the Christmas spirit, making a much better metaverse avatar than its fellow Meta, too.
Nvidia CEO Jensen Huang has drunk his own company’s ubiquitous juice and become a “toy” for the online world.
“Since I’m too shy to sing myself, I asked a friend to help me,” Huang explains in the video embedded below.
It’s called ‘Toy Jensen’ and yes, it’s as weird as it sounds.
But it’s also pretty remarkable what Nvidia has accomplished.
“Toy Jensen, TJ, is an AI avatar, he’s fully interactive, makes eye contact, has conversations about many topics, speaks and sings with my voice, and animates his face and body based on what he’s trying to say” , Huang said.
TJ is built on Nvidia’s AI, Omniverse, and RTX technologies, and the real point of all of this is to be a technology demo that showcases core technologies that are part of the Nvidia Omniverse Avatar Cloud Engine, or ACE, platform.
Omniverse ACE is a collection of cloud-native AI workflows and microservices for developers to create, customize, and deploy avatars that can interact and interact with the viewer.
To make this happen, the Nvidia team used a newly developed voice conversion model to extract a professional singer’s voice from a sample provided by them and convert it into TJ’s voice, originally developed using training hours of real-world recordings. They used the musical notes from this sample and applied them to TJ’s digital voice to make the avatar sing the same notes and rhythm as the original singer.
The performance is superb and the song itself doesn’t sound too bad. Watch out, Michael Bublé, my mom would eat this.
Also, Toy Jensen has legs.