Nav: Home

Novel software can recognize eye contact in everyday situations

August 11, 2017

"Until now, if you were to hang an advertising poster in the pedestrian zone, and wanted to know how many people actually looked at it, you would not have had a chance", explains Andreas Bulling, who leads the independent research group "Perceptual User Interfaces" at the Excellence Cluster at Saarland University and the Max Planck Institute for Informatics. Previously, one would try to capture this important information by measuring gaze direction. This required special eye tracking equipment which needed minutes-long calibration; what was more, everyone had to wear such a tracker. Real-world studies, such as in a pedestrian zone, or even just with multiple people, were in the best case very complicated and in the worst case, impossible.

Even when the camera was placed at the target object, for example the poster, and machine learning was used i.e. the computer was trained using a sufficient quantity of sample data only glances at the camera itself could be recognized. Too often, the difference between the training data and the data in the target environment was too great. A universal eye contact detector, usable for both small and large target objects, in stationary and mobile situations, for one user or a whole group, or under changing lighting conditions, was hitherto nearly impossible.

Together with his PhD student Xucong Zhang, and his former PostDoc Yusuke Sugano, now a Professor at Osaka University, Bulling has developed a method [1] that is based on a new generation of algorithms for estimating gaze direction. These use a special type of neural network, known as "Deep Learning", that is currently creating a sensation in many areas of industry and business. Bulling and his colleagues have already been working on this approach for two years [2] and have advanced it step by step [3]. In the method they are now presenting, first a so-called clustering of the estimated gaze directions is carried out. With the same strategy, one can, for example, also distinguish apples and pears according to various characteristics, without having to explicitly specify how the two differ. In a second step, the most likely clusters are identified, and the gaze direction estimates they contain are used for the training of a target-object-specific eye contact detector. A decisive advantage of this procedure is that it can be carried out with no involvement from the user, and the method can also improve further, the longer the camera remains next to the target object and records data. "In this way, our method turns normal cameras into eye contact detectors, without the size or position of the target object having to be known or specified in advance," explains Bulling.

The researchers have tested their method in two scenarios: in a workspace, the camera was mounted on the target object, and in an everyday situation, a user wore an on-body camera, so that it took on a first-person perspective. The result: Since the method works out the necessary knowledge for itself, it is robust, even when the number of people involved, the lighting conditions, the camera position, and the types and sizes of target objects vary.

However, Bulling notes that "we can in principle identify eye contact clusters on multiple target objects with only one camera, but the assignment of these clusters to the various objects is not yet possible. Our method currently assumes that the nearest cluster belongs to the target object, and ignores the other clusters. This limitation is what we will tackle next." He is nonetheless convinced that "the method we present is a great step forward. It paves the way not only for new user interfaces that automatically recognize eye contact and react to it, but also for measurements of eye contact in everyday situations, such as outdoor advertising, that were previously impossible."

Further Information:
[1] Xucong Zhang, Yusuke Sugano and Andreas Bulling. Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery. Proc. ACM UIST 2017.

[2] Xucong Zhang, Yusuke Sugano, Mario Fritz and Andreas Bulling. Appearance-Based Gaze Estimation in the Wild. Proc. IEEE CVPR 2015, 4511-4520.

[3] Xucong Zhang, Yusuke Sugano, Mario Fritz and Andreas Bulling. It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation. Proc. IEEE CVPRW 2017.

Demo Video:

Questions can be directed to:

Dr. Andreas Bulling
Perceptual User Interfaces Group
Cluster of Excellence "Multimodal Computing and Interaction"
Saarland Informatics Campus
Tel. 49-681-932-52128


Gordon Bolduan
Competence Center Computer Science Saarland
Saarland Informatics Campus
Phone: 49-681-302-70741

Notice for Radio Journalists:

You can conduct telephone interviews in studio quality with scientists of Saarland University over the Radio Codec (IP connection with direct dialing or via ARD Sternpunkt 106813020001). Please send interview requests to the press office (0681/302-3610).

Saarland University

Related Eye Contact Articles:

Massachusetts General researchers explore why those with autism avoid eye contact
Individuals with autism spectrum disorder often find it difficult to look others in the eyes as they find eye contact uncomfortable or stressful.
Disfiguring eye symptoms diminish in Graves' eye disease drug trial
Graves' eye disease, also known as thyroid eye disease, takes a physical and emotional toll on patients as inflammation and tissue buildup cause the eyes to bulge painfully from their sockets.
Toddlers with autism don't avoid eye contact, but do miss its significance
Marcus Autism Center researchers studied eye movements in 2-year-olds with and without autism.
Drug-dispensing contact lens effectively lowers eye pressure in glaucoma model
In a study published online today in Ophthalmology, a team of researchers have shown that a novel contact lens-based system, which uses a strategically placed drug polymer film to deliver medication gradually to the eye, is at least as effective, and possibly more so, as daily latanoprost eye drops in a preclinical model for glaucoma.
Increased eye cancer risk linked to pigmentation genes that dictate eye color
New research links specific inherited genetic differences to an increased risk for eye (uveal) melanoma, a rare form of melanoma that arises from pigment cells that determine eye color. scientists report the first evidence of a strong association between genes linked to eye color and development of uveal melanoma.
Hospital more than 4 times as likely for under-6s after laundry pod detergent contact
Children under the age of 6 are four times as likely to end up in hospital after contact with laundry pod detergent as children exposed to other types of detergent, finds research published online in the journal Injury Prevention.
In corneal transplantation, men and women don't see eye to eye
A study of patients undergoing corneal transplants indicates that subtle differences between men and women may lead to poorer outcomes for a woman who has received a cornea from a male donor.
Equilibrium modeling increases contact lens comfort
In an article publishing this week in the SIAM Journal on Applied Mathematics, authors David Ross, Kara Maki, and Emily Holz design an equilibrium model to demonstrate the elastic stresses and suction pressure distribution between a soft hydrogel contact lens and an eye.
USC Roski Eye Institute researchers publish largest eye study among Latinos
The University of Southern California Roski Eye Institute researchers and clinicians published results of the largest population-based study of adult Latinos and age-related macular degeneration (AMD) in the National Eye Institute-funded 'Los Angeles Latino Eye Study (LALES).' The study, published in JAMA Ophthalmology, is the first to analyze the risk and prevalence of early and late stage AMD and its impact on quality of life for older Latinos.
USC Roski Eye Institute researchers publish largest Chinese American eye study
USC Roski Eye Institute researchers publish largest eye study among Chinese Americans in JAMA Ophthalmology identifying more effective ways to prevent and treat blinding eye diseases for this racial group.

Related Eye Contact Reading:

The Power of Eye Contact: Your Secret for Success in Business, Love, and Life
by Michael Ellsberg (Author)

Eye contact can land you a job. It can get you a date. It can deepen your connections with the people you love. It can make or break business relationships. It can help win a fight. It can win over an audience.

Simply put, eye contact is one of the most powerful tools in human face-to-face interaction. The Power of Eye Contact is your concise guide to harnessing the potent force of eye contact.

Master this force and you will notice three things:

You meet more people. Your connections deepen with family, friends, and business prospects. You look, feel, and act more... View Details

Eye Contact
by Cammie McGovern (Author)

Read Cammie McGovern's posts on the Penguin Blog

Like The Lovely Bones and The Curious Incident of the Dog in the Night-Time, Cammie McGovern’s breakout novel is at once a hypnotic thriller and an affecting portrait of people as real as our next-door neighbors. In Eye Contact, two children vanish in the woods behind their elementary school. Hours later, nine-year-old Adam is found alive, the sole witness to his playmate’s murder. But because Adam has autism, he is a silent witness. Only his mother, Cara, can help decode his behavior for the police. As... View Details

Colored Contact Lenses: an Easy Consumer's Guide

This Kindle Book directs the consumer's buying behaviors of colored contact lenses to the best, for life.

You will enjoy reading professional reviews of the colored contact lenses brands with me (the author of the book, Dr. Rami Baramada).

I am an optometrist who has many years of editorial experience in consumer publishing, with an emphasis on the health and beauty fields.

I had published some articles to educate women and consumers about the colored contact lenses. I didn't expect the huge useful impact that these articles had left on the readers. I had got... View Details

Superhuman Eye Contact: How to Radiate Confidence, Attract Others, and Demand Re
by Patrick King (Author)

Do you struggle to hold eye contact? Does it make you feel unconfident, uncomfortable, and self-conscious? Or do you just want to appear more charismatic and confident? Then Superhuman Eye Contact has the insightful tips and innovative exercises you need to become an eye contact expert – overnight. If you are bad at eye contact, people will assume that you are creepy or untrustworthy. If you are merely average at eye contact, you won’t be negative, yet you won’t be memorable either. But if you are SUPERHUMAN at eye contact, you will instantly make an impression and have people... View Details

Eye Contact: Social Networking, Face to Face, With a Camera
by Max James Fallon (Photographer)

It will not be easy, but it will be worth the effort: turn off that iPhone, close your laptop, shut down your tablet, turn off the TV… Then grab your camera if you want to really connect with someone who isn’t a digitized ephemeral image, floating on your screen, a real human being, face-to-face! That’s what Max James Fallon suggests in his new book, EYE CONTACT: Social Networking {Face-to-Face} With a Camera. He’s one curious guy when it comes to meeting people when he travels, and to satisfy our need to connect and interact with our fellow human beings, he uses his camera to... View Details

Eye Contact Training: Learn How To Attract Women + Improve Your Self Confidence, Charisma, & Leadership
by Robert Moore (Author)

Attract Women with the Power of Your Gaze!

Would you like to be more confident? Do you wish you could hold a woman’s attention longer? Is it hard to look women in the eyes?

When you purchase Robert Moore’s Eye Contact Training: Learn How to Attract Women + Improve Your Self Confidence, Charisma & Leadership, you’ll gain access to a massive toolbox of proven tips and strategies for dating success. These eye-contact training exercises can dramatically improve your inner game - and your level of success with women!

Do you want to be an “alpha male”?... View Details

Cure Your Shy Eye: Your Practical Guide To Master Eye Contact In 21 Days
by Sagar Takker (Author)

Are you comfortable making eye contact with anyone? Do you want to be able to do that? Then read Cure Your Shy Eye: Your Practical Guide To Master Eye Contact In 21 Days.
1. This is a practical book on improving eye contact. It contains only relevant and necessary information on developing the skill and habit of making eye contact. It is not about the history of eye contact, it's importance, benefits or any of that academic information. If the reader needs such information or needs to be convinced of the importance of making eye contact, this book won't be very helpful for that... View Details

Improve Your Vision Without Glasses or Contact Lenses
by Steven M. Beresford (Author), David W. Muris (Author), Merril J. Allen (Author), Francis A. Young (Author)

For the millions of Americans who suffer some type of vision problem, this complete course in vision therapy helps increase focusing power, decrease eyestrain, and prevent further deterioration of vision. View Details

Eye Contact (The Conditioned Book 1)
by Rebecca Royce

He knows she can't be trusted,
But he can't resist the temptation.

Burdened with a dirty little secret, Addison Wade refuses to let anyone in--especially her family. But when her nephew goes missing, Addison will risk her own life to save him from a fate far worse than death.

Spencer Lewis can find anyone--at any time. And after one look at Addison, the granddaughter of the most powerful man in the country, he can't possibly turn her away. She needs him. But teaming up with her may be more dangerous than he ever realized. As Spencer and Addison search for the... View Details

Eye Contact: The Power of One on One
by Simon & Schuster

A Simon & Schuster eBook. Simon & Schuster has a great book for every reader. View Details

Best Science Podcasts 2017

We have hand picked the best science podcasts for 2017. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Simple Solutions
Sometimes, the best solutions to complex problems are simple. But simple doesn't always mean easy. This hour, TED speakers describe the innovation and hard work that goes into achieving simplicity. Guests include designer Mileha Soneji, chef Sam Kass, sleep researcher Wendy Troxel, public health advocate Myriam Sidibe, and engineer Amos Winter.
Now Playing: Science for the People

#448 Pavlov (Rebroadcast)
This week, we're learning about the life and work of a groundbreaking physiologist whose work on learning and instinct is familiar worldwide, and almost universally misunderstood. We'll spend the hour with Daniel Todes, Ph.D, Professor of History of Medicine at The Johns Hopkins University, discussing his book "Ivan Pavlov: A Russian Life in Science."