Disney Research method captures facial details at high fidelity and real time

August 05, 2015

Forehead wrinkles that rapidly deepen and crow's feet that appear suddenly around the eyes might distress the average person, but the ability to quickly and realistically incorporate such details in a facial reconstruction is the key feature of a new performance capture method developed at Disney Research.

The method, which requires only a single video camera such as a webcam, is the first to both operate in real-time and to capture facial features in high resolution, including such details as wrinkles.

"This could open up a variety of new applications, from casual home users who wish to give their own likeness to a game character to commercial applications where, for example, a customer can apply and view virtual makeup," said Derek Bradley, research scientist at Disney Research. "In film production, our method would allow a director to assess an actor's performance transferred to a digital creature even as he is filming."

In addition to Bradley, the research team included Chen Cao and Kun Zhou of Zhejiang University as well as Thabo Beeler of Disney Research. The researchers will present their facial performance capture method at ACM SIGGRAPH 2015, the International Conference on Computer Graphics and Interactive Techniques, in Los Angeles Aug. 9-13.

Capturing facial details has proven critical for conveying emotions or the state of mind of a character and audiences are sensitive to subtle characteristics of the face. Animators thus devote great care to tracking facial performances of actors so that they can create realistic virtual characters. Until now, these facial performance capture methods have been designed to either capture these details in high resolution offline, or to record less detail but at high speed.

The new method represents a substantial step towards the goal of attaining both high quality and high speed.

As with other real-time facial capture methods, the researchers begin by using generic, low-resolution face models as the basis for reconstruction. A face model is virtually fitted to the user by the system as the user changes expressions and contorts his or her face in front of a video camera.

The system then automatically calculates a personalized wrinkle probability map, based on the set of extreme expressions the user assumed to train the program. The wrinkles may not match the user's actual wrinkles exactly, but they exhibit a very similar appearance because of the way they are formed.

"By learning the relationship between local image appearance and local wrinkle formation, we can reconstruct plausible face wrinkles in real-time from a single camera," Beeler noted. Once trained, the online capture approach can be applied to any new user without additional training. The system is generic and can be used with most common types of real-time face trackers.
For more information on the method, and a video, please visit the project web page at http://www.disneyresearch.com/realtimeperformancecapture/.

About Disney Research

Disney Research is a network of research laboratories supporting The Walt Disney Company. Its purpose is to pursue scientific and technological innovation to advance the company's broad media and entertainment efforts. Vice Presidents Jessica Hodgins and Markus Gross manage Disney Research facilities in Los Angeles, Pittsburgh, Zürich, and Boston and work closely with the Pixar and ILM research groups in the San Francisco Bay Area. Research topics include computer graphics, animation, video processing, computer vision, robotics, wireless & mobile computing, human-computer interaction, displays, behavioral economics, and machine learning.

Disney Research

Related Wrinkles Articles from Brightsurf:

The benefits of slowness
Wrinkles, furrows, spots: a person's aging process is accompanied by tell-tale signs on their face.

Scientists iron out the physics of wrinkling
In a paper recently published in Applied Physics Letters, researchers from the Okinawa Institute of Science and Technology Graduate University (OIST) have shown how wrinkles can be increased or reduced by altering the curvature at the edge of a material.

Study shows biocell collagen ingestion reduced signs of UVB-induced photoaging
New research finds BioCell Collagen Ingestion to reduce signs of UVB-Induced photoaging, which accounts for a significant amount of visible skin damage.

Mechanical forces shape bacterial biofilms' puzzling patterns
Belying their slimy natures, the sticky patches of bacteria called biofilms often form intricate, starburst-like patterns as they grow.

Express yourself: Dermal fillers restore youthful facial movement, don't just fill wrinkles
Used for facial rejuvenation, dermal fillers do more than just fill in wrinkles.

WSU genetic discovery holds implications for better immunity, longer life
Wrinkles on the skin of a microscopic worm might provide the key to a longer, healthier life for humans.

New silk materials can wrinkle into detailed patterns, then unwrinkle to be 'reprinted'
Tufts engineers have developed silk materials that can wrinkle into highly detailed patterns -- including words, textures and images as intricate as a QR code or a fingerprint.

Smoothing wrinkles in mice -- without needles
In the quest for a more youthful appearance, many people slather ointments on their skin or undergo injections of dermal fillers.

Study shows BioCell collagen can visibly reduce common signs of skin aging within 12 weeks
In one of the most substantial studies of a skin health supplement, BioCell Collagen®, was found to visibly reduce common signs of skin aging, including lines and wrinkles, within 12 weeks of daily use.

Puzzling shapes: Unlocking the mysteries of plant cell morphology
The discovery of the mechanics and molecular mechanism that dictate cell shape formation in plants by a team of McGill researchers offers new clues about the fundamental processes governing tissue formation in multicellular organisms.

Read More: Wrinkles News and Wrinkles Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.