Computer-generated special effects have amazed audiences for nearly a generation now. But they’ve long fallen short when modeling the one thing that’s most important to telling a good story: the human face. Until now.
At our annual GPU Technology Conference this week, Jen-Hsun Huang, our CEO, showed off a startling digital model of a human head that not only offered unparalleled realism — but was generated in real-time using commercially-available GPUs and an NVIDIA innovation we call FaceWorks.
“Let’s have Ira say a few things,” Jen-Hsun asks during his keynote presentation, as Ira’s visage loomed overhead. “What did you have for breakfast?”
Ira complains that he had yogurt for breakfast, but it was largely frozen fruit.
Asked about Project SHIELD, Ira screams “take my money!”
To create ‘Ira’ we partnered with Dr. Paul Debevec of the Institute for Creative Technology (ICT) at the University of Southern California. He and his team have been building next-generation systems that can capture facial data to within a tenth of a millimeter without the need for special makeup, awkward markers or specialized cameras.
ICT’s ‘light stage’ technology uses photographic techniques to derive not only the 3D shape of an actor’s face, but the critical elements needed to properly represent human skin. These include light reflection and transmission through the skin, reflectivity from oils in the skin, and the nearly microscopic lines and bumps in the skin surface.
After asking an actor to model several dozen basic expressions, Debevec’s team then runs their footage through a compression engine. The result: a pared down set of data that can be mixed and matched to generate a wide range of expressions.
Those interested can download it here.