Nav: Home

The relationship between looking/listening and human emotions

June 19, 2020


A research team from the Department of Computer Science and Engineering and the Electronics-Inspired Interdisciplinary Research Institute at Toyohashi University of Technology has indicated that the relationship between attentional states in response to pictures and sounds and the emotions elicited by them may be different in visual perception and auditory perception. This result was obtained by measuring pupillary reactions related to human emotions. It suggests that visual perception elicits emotions in all attentional states, whereas auditory perception elicits emotions only when attention is paid to sounds, thus showing the differences in the relationships between attentional states and emotions in response to visual and auditory stimuli.


In our daily lives, our emotions are often elicited by the information we receive from visual and auditory perception. As such, many studies up until now have investigated human emotional processing using emotional stimuli such as pictures and sounds. However, it was not clear whether such emotional processing differed between visual and auditory perception.

Our research team asked participants in the experiment to perform four tasks to alert them to various attentional states when they were presented with emotionally arousing pictures and sounds, in order to investigate how emotional responses differed between visual and auditory perception. We also compared the pupillary responses obtained by eye movement measurements as a physiological indicator of emotional responses. As a result, visual perception (pictures) elicited emotions during the execution of all tasks, whereas auditory perception (sounds) did so only during the execution of tasks where attention was paid to the sounds. These results suggest that there are differences in the relationship between attentional states and emotional responses to visual and auditory stimuli.

"Traditionally, subjective questionnaires have been the most common method for assessing emotional states. However, in this study, we wanted to extract emotional states while some kind of task was being performed. We therefore focused on pupillary response, which is receiving a lot of attention as one of the biological signals that reflect cognitive states. Although many studies have reported about attentional states during emotional arousal owing to visual and auditory perception, there have been no previous studies comparing these states across senses, and this is the first attempt", explains the lead author, Satoshi Nakakoga, Ph. D. student.

Besides, Professor Tetsuto Minami, the leader of the research team, said, "There are more opportunities to come into contact with various visual media via smartphones and other devices and to evoke emotions through that visual and auditory information. We will continue investigating about sensory perception that elicits emotions, including the effects of elicited emotions on human behavior."

Future Outlook

Based on the results of this research, our research team indicates the possibility of a new method of emotion regulation in which the emotional responses elicited by a certain sense are promoted or suppressed by stimuli input from another sense. Ultimately, we hope to establish this new method of emotion regulation to help treat psychiatric disorders such as panic and mood disorders.

Nakakoga, S., Higashi, H., Muramatsu, J., Nakauchi, S., & Minami, T. (2020). Asymmetrical characteristics of emotional responses to pictures and sounds: Evidence from pupillometry. PLoS ONE, 15(4), e0230775. doi: 10.1371/journal.pone.0230775

Toyohashi University of Technology (TUT)

Related Emotions Articles:

Why are memories attached to emotions so strong?
Multiple neurons in the brain must fire in synchrony to create persistent memories tied to intense emotions, new research from Columbia neuroscientists has found.
The relationship between looking/listening and human emotions
Toyohashi University of Technology has indicated that the relationship between attentional states in response to pictures and sounds and the emotions elicited by them may be different in visual perception and auditory perception.
Multitasking in the workplace can lead to negative emotions
From writing papers to answering emails, it's common for office workers to juggle multiple tasks at once.
Do ER caregivers' on-the-job emotions affect patient care?
Doctors and nurses in emergency departments at four academic centers and four community hospitals in the Northeast reported a wide range of emotions triggered by patients, hospital resources and societal factors, according to a qualitative study led by a University of Massachusetts Amherst social psychologist.
The 'place' of emotions
The entire set of our emotions is mapped in a small region of the brain, a 3 centimeters area of the cortex, according to a study conducted at the IMT School for Advanced Studies Lucca, Italy.
Faking emotions at work does more harm than good
Faking your emotions at work to appear more positive likely does more harm than good, according to a University of Arizona researcher.
Students do better in school when they can understand, manage emotions
Students who are better able to understand and manage their emotions effectively, a skill known as emotional intelligence, do better at school than their less skilled peers, as measured by grades and standardized test scores, according to research published by the American Psychological Association.
How people want to feel determines whether others can influence their emotions
New Stanford research on emotions shows that people's motivations are a driving factor behind how much they allow others to influence their feelings, such as anger.
Moral emotions, a diagnotic tool for frontotemporal dementia?
A study conducted by Marc Teichmann and Carole Azuar at the Brain and Spine Institute in Paris (France) and at the Pitié-Salpêtrière Hospital shows a particularly marked impairment of moral emotions in patients with frontotemporal dementia (FTD).
Emotions from touch
Touching different types of surfaces may incur certain emotions. This was the conclusion made by the psychologists from the Higher School of Economics in a recent empirical study.
More Emotions News and Emotions Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Power Of Spaces
How do spaces shape the human experience? In what ways do our rooms, homes, and buildings give us meaning and purpose? This hour, TED speakers explore the power of the spaces we make and inhabit. Guests include architect Michael Murphy, musician David Byrne, artist Es Devlin, and architect Siamak Hariri.
Now Playing: Science for the People

#576 Science Communication in Creative Places
When you think of science communication, you might think of TED talks or museum talks or video talks, or... people giving lectures. It's a lot of people talking. But there's more to sci comm than that. This week host Bethany Brookshire talks to three people who have looked at science communication in places you might not expect it. We'll speak with Mauna Dasari, a graduate student at Notre Dame, about making mammals into a March Madness match. We'll talk with Sarah Garner, director of the Pathologists Assistant Program at Tulane University School of Medicine, who takes pathology instruction out of...
Now Playing: Radiolab

What If?
There's plenty of speculation about what Donald Trump might do in the wake of the election. Would he dispute the results if he loses? Would he simply refuse to leave office, or even try to use the military to maintain control? Last summer, Rosa Brooks got together a team of experts and political operatives from both sides of the aisle to ask a slightly different question. Rather than arguing about whether he'd do those things, they dug into what exactly would happen if he did. Part war game part choose your own adventure, Rosa's Transition Integrity Project doesn't give us any predictions, and it isn't a referendum on Trump. Instead, it's a deeply illuminating stress test on our laws, our institutions, and on the commitment to democracy written into the constitution. This episode was reported by Bethel Habte, with help from Tracie Hunte, and produced by Bethel Habte. Jeremy Bloom provided original music. Support Radiolab by becoming a member today at     You can read The Transition Integrity Project's report here.