Turning photos into an interactive experience

November 27, 2017

BANGKOK, Thailand, DATE TK--Human facial expressions, big and small, have the ability to convey what a person is feeling. Imagine being able to bring out a wide range of human emotions in any still photo, and to do so, automatically.

Computer scientists at Tel-Aviv University collaborated with researchers at Facebook to develop a new computational technique that makes it possible for users to animate still images in a highly realistic way. The method enables subjects in a still photo to come to life and express various emotions. For instance, when a user "likes" a photo on Facebook or tweets a thumbs-up, the facial expression of his or her profile photo could automatically react with a smile or express an emotion of approval or happiness. The researchers demonstrate their technique on varying facial images, including selfies, old portraits and facial avatars. Their results were able to show portraits brought to life, depicting a person in the photo as though they were actually breathing, for example, or smiling or frowning.

"Facial expressions in humans convey not just major emotions, but through subtle variations, a rather nuanced view into the emotional state of a person," said Hadar Averbuch-Elor, computer science PhD at Tel-Aviv University and a lead author of the study. "This expressiveness is what we attempt to capture and animate in our work."

Coauthors of the research, "Bringing Portraits to Life," include Daniel Cohen-Or, professor of computer science at Tel-Aviv University, and Facebook research scientists Michael F. Cohen and Johannes Kopf. The team will present their innovative work at SIGGRAPH Asia 2017 in Bangkok, 27 November to 30 November. This annual conference and exhibition showcases the world's leading professionals, academics and creative minds at the forefront of computer graphics and interactive techniques.

Given a single image, the researchers' method automatically generates photo-realistic video clips that express various emotions. Previous facial animation techniques typically assume the availability of a video of the target face, which exhibits variation in pose and expression. In this work, the researchers use as input only a single image of a target face to animate it.

"This makes our method more widely applicable, to the near endless supply of portraits or selfies on the Internet," explained Cohen. "We animate the single target face image from a driving video, allowing the target image--the still photo--to come alive and mimic the expressiveness of the subject in the driving video."

To this end, the target image in the still photo is animated by a series of geometric warps that imitate the facial transformations in the driving video. Previous work has so far been restricted to animating only the face region but, within limits, this new method can animate the full head and upper body. The researchers manipulate the face by lightweight 2D warps and are able to create moderate head movement that is depicted very realistically without having to convert or project the image to 3D. Technically, the main challenge, according to the researchers, was to preserve the identity of the person's face in the still photo while manipulating it with warps and features taken and transferred from frames of a driving video.

The new method also includes continuous transfer of fine-scale details, such as facial wrinkles and creases while avoiding so-called "outlier" wrinkles that are caused by cast shadows or misalignment between the warped video frames. When needed, their computational technique can also automatically fill in "hidden" facial details. For instance, the interior of the mouth can be depicted even if the person in the target photo has his/her mouth closed.

In a user study, participants were presented with 24 randomly selected videos, eight of which were real. They were asked to rate them as real or not real based on how the animation appeared meaning 50 percent would be a perfect score. The researchers' animated videos were identified as real 46 percent of the time. The "happy" animations were perceived as the most real (identified as real 58 percent of the time), and the "surprise" ones perceived as less real (identified as real 37 percent of the time).

In the future, the team intends to build on this work by combining their technique with 3D methods or coupling it with artificial intelligence to create an interactive avatar starting from a single photograph.
About SIGGRAPH Asia 2017

The 10th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia will take place in Bangkok, Thailand at the at the Bangkok International Trade and Exhibition Centre (BITEC) from 27 - 30 November 2017. The annual event held in Asia attracts the most respected technical and creative people from all over the world who are excited by research, science, art, animation, gaming, interactivity, education and emerging technologies. The four-day SIGGRAPH Asia 2017 conference includes a diverse range of juried programs, such as the Art Gallery, Computer Animation Festival, Courses, Emerging Technologies, Posters, Symposium on Education, Symposium on Mobile Graphics and Interactive Applications, Symposium on Visualization, Technical Briefs, Technical Papers, VR Showcase and Workshops. A three-day exhibition held from 28 - 30 November 2017 will offer a business platform for industry players to market their innovative products and services to the computer graphics and interactive techniques professionals and enthusiasts from Asia and beyond.


The Association of Computing Machinery's Special Interest Group on Computer Graphics and Interactive Techniques (ACM SIGGRAPH) sponsors SIGGRAPH Asia 2017. Founded in 1947, ACM is an educational and scientific society uniting the world's computing educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM strengthens the profession's collective voice through strong leadership, promotion of the highest standards, and recognition of technical excellence. The ACM SIGGRAPH deals with all aspects of graphical user/computer communication and manipulation: hardware, languages, data structure, methodology, and applications

Association for Computing Machinery

Related Emotions Articles from Brightsurf:

Why are memories attached to emotions so strong?
Multiple neurons in the brain must fire in synchrony to create persistent memories tied to intense emotions, new research from Columbia neuroscientists has found.

The relationship between looking/listening and human emotions
Toyohashi University of Technology has indicated that the relationship between attentional states in response to pictures and sounds and the emotions elicited by them may be different in visual perception and auditory perception.

Multitasking in the workplace can lead to negative emotions
From writing papers to answering emails, it's common for office workers to juggle multiple tasks at once.

Do ER caregivers' on-the-job emotions affect patient care?
Doctors and nurses in emergency departments at four academic centers and four community hospitals in the Northeast reported a wide range of emotions triggered by patients, hospital resources and societal factors, according to a qualitative study led by a University of Massachusetts Amherst social psychologist.

The 'place' of emotions
The entire set of our emotions is mapped in a small region of the brain, a 3 centimeters area of the cortex, according to a study conducted at the IMT School for Advanced Studies Lucca, Italy.

Faking emotions at work does more harm than good
Faking your emotions at work to appear more positive likely does more harm than good, according to a University of Arizona researcher.

Students do better in school when they can understand, manage emotions
Students who are better able to understand and manage their emotions effectively, a skill known as emotional intelligence, do better at school than their less skilled peers, as measured by grades and standardized test scores, according to research published by the American Psychological Association.

How people want to feel determines whether others can influence their emotions
New Stanford research on emotions shows that people's motivations are a driving factor behind how much they allow others to influence their feelings, such as anger.

Moral emotions, a diagnotic tool for frontotemporal dementia?
A study conducted by Marc Teichmann and Carole Azuar at the Brain and Spine Institute in Paris (France) and at the Pitié-Salpêtrière Hospital shows a particularly marked impairment of moral emotions in patients with frontotemporal dementia (FTD).

Emotions from touch
Touching different types of surfaces may incur certain emotions. This was the conclusion made by the psychologists from the Higher School of Economics in a recent empirical study.

Read More: Emotions News and Emotions Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.