Why music makes us feel, according to AI

October 31, 2019

In a new paper, a team of USC computer scientists and psychologists teamed up to investigate how music affects how you act, feel and think

Your heart beats faster, palms sweat and part of your brain called the Heschl's gyrus lights up like a Christmas tree. Chances are, you've never thought about what happens to your brain and body when you listen to music in such a detailed way.

But it's a question that has puzzled scientists for decades: Why does something as abstract as music provoke such a consistent response? In a new study, a team of USC researchers, with the help of artificial intelligence, investigated how music affects listeners' brains, bodies and emotions.

The research team looked at heart rate, galvanic skin response (or sweat gland activity), brain activity and subjective feelings of happiness and sadness in a group of volunteers as they listened to three pieces of unfamiliar music.

Of the 74 musical features examined, the researchers found dynamics, register, rhythm and harmony were particularly helpful in predicting listeners' response.

"Taking a holistic view of music perception, using all different kinds of musical predictors, gives us an unprecedented insight into how our bodies and brains respond to music," said the study's lead author Tim Greer, a computer science PhD student and a member of the USC Signal Analysis and Interpretation Laboratory (SAIL).

Contrast is crucial

Amongst their findings, the researchers noted that music powerfully influenced parts of the brain in the auditory complex called the Heschls' gyrus and the superior temporal gyrus (plural: gyri).

Specifically, the brain responded to pulse clarity, or the strength of the beat (put simply: your gyri will be looking lively when listening to Lady Gaga's Bad Romance).

They also found that changing dynamics, rhythm and timbre, or the introduction of new instruments, brings about the uptick in response. In other words, contrast is crucial. For instance, the gyri activate when there is a change in dynamics, or "loudness."

"If a song is loud throughout, there's not a lot of dynamic variability, and the experience will not be as powerful as if the composer uses a change in loudness," said Greer, himself a composer who plays sax and keyboard.

"It's the songwriter's job to take you on a rollercoaster of emotions in under three minutes, and dynamic variability is one of the ways this is achieved."

So, if you're listening to a whole album of black metal, which is consistently loud, you're probably not going to see a response. But if you're listening to Nirvana's Smells Like Teen Spirit, which goes from a quiet verse to loud chorus and back again, it's a different story.

The team also discovered that galvanic skin response--basically, a measure of sweat -- increased after the entrance of a new instrument or start of a musical crescendo.

"When each new instrument enters, you can see a spike in the collective response of the skin," said Greer.

In addition, the most stimulating moments in music were preceded by an increase in the complexity level of the song. In essence, the more instruments are in the song, the more people responded. (Think: the first section of Mike Oldfield's Tubular Bells, as the song builds to a crescendo by adding more instruments.)

And the saddest note of all? That award goes to the raised 7th note of the minor scale. The study found the note F# in a song in G minor key of G minor positively correlated with high sadness ratings.

That might be the reason the narrator's anguish is almost palpable in The Animals' House of the Rising Sun, which uses the raised 7th of the minor scale to launch into each increasingly emotional verse.

New territory

For this experiment, the team selected three emotional pieces of music that did not contain lyrics and were not highly familiar, so no element of memory was attached to the listeners' emotional response. (Hearing a song that played in the background during a wisdom tooth extraction, for instance, might skew your perception.)

In the neuroimaging experiment, 40 volunteers listened to a series of sad or happy musical excerpts, while their brains were scanned using MRI. This was conducted at USC's Brain and Creativity Institute by Assal Habibi, an assistant professor of psychology at USC Dornsife College of Letters, Arts and Sciences, and her team, including Matthew Sachs, a postdoctoral scholar currently at Columbia University.

To measure physical reaction, 60 people listened to music on headphones, while their heart activity and skin conductance were measured. The same group also rated the intensity of emotion (happy or sad) from 1 to 10 while listening to the music.

Then, the computer scientists crunched the data using AI algorithms to determine which auditory features people responded to consistently.

In the past, neuroscientists trying to better understand the impact of music on the body, brain and emotions have analyzed MRI brain scans over very short segments of time--for instance, looking at the brain reacting to two seconds of music.

By contrast, in this study, using algorithms to analyze data gathered in the lab, the scientists were able to look at how people felt while listening to music over longer periods of time, not only from brain scans, but also combining data from other modes.

"Novel multimodal computing approaches help not just illuminate human affective experiences to music at the brain and body level, but in connecting them to how actually individuals feel and articulate their experiences," said Professor Shrikanth (Shri) Narayanan, study co-author, Niki and C. L. Max Nikias Chair in Engineering and professor of electrical and computer engineering and computer science.

Feeling good

In addition to helping researchers identify songs for the perfect workout, study or sleep playlist, the research has therapeutic applications -- music has been shown to calm anxiety, ease pain and help people with disabilities or dementia.

"From a therapy perspective, music is a really good tool to induce emotion and engage a better mood," said Habibi.

"Using this research, we can design musical stimuli for therapy in depression and other mood disorders. It also helps us understand how emotions are processed in the brain."

According to the researchers, future studies could look at how different types of music can positively manipulate our emotional responses and whether the intent of the composer matches the listener's perception of a piece of music.
The paper, titled "A Multimodal View into Music's Effect on Human Neural, Physiological, and Emotional Experience," was presented at ACM Multimedia, Oct. 22. The research team also includes Ben Ma, a USC Viterbi School of Engineering undergraduate student in computer science and a member of USC SAIL.

University of Southern California

Related Brain Articles from Brightsurf:

Glioblastoma nanomedicine crosses into brain in mice, eradicates recurring brain cancer
A new synthetic protein nanoparticle capable of slipping past the nearly impermeable blood-brain barrier in mice could deliver cancer-killing drugs directly to malignant brain tumors, new research from the University of Michigan shows.

Children with asymptomatic brain bleeds as newborns show normal brain development at age 2
A study by UNC researchers finds that neurodevelopmental scores and gray matter volumes at age two years did not differ between children who had MRI-confirmed asymptomatic subdural hemorrhages when they were neonates, compared to children with no history of subdural hemorrhage.

New model of human brain 'conversations' could inform research on brain disease, cognition
A team of Indiana University neuroscientists has built a new model of human brain networks that sheds light on how the brain functions.

Human brain size gene triggers bigger brain in monkeys
Dresden and Japanese researchers show that a human-specific gene causes a larger neocortex in the common marmoset, a non-human primate.

Unique insight into development of the human brain: Model of the early embryonic brain
Stem cell researchers from the University of Copenhagen have designed a model of an early embryonic brain.

An optical brain-to-brain interface supports information exchange for locomotion control
Chinese researchers established an optical BtBI that supports rapid information transmission for precise locomotion control, thus providing a proof-of-principle demonstration of fast BtBI for real-time behavioral control.

Transplanting human nerve cells into a mouse brain reveals how they wire into brain circuits
A team of researchers led by Pierre Vanderhaeghen and Vincent Bonin (VIB-KU Leuven, Université libre de Bruxelles and NERF) showed how human nerve cells can develop at their own pace, and form highly precise connections with the surrounding mouse brain cells.

Brain scans reveal how the human brain compensates when one hemisphere is removed
Researchers studying six adults who had one of their brain hemispheres removed during childhood to reduce epileptic seizures found that the remaining half of the brain formed unusually strong connections between different functional brain networks, which potentially help the body to function as if the brain were intact.

Alcohol byproduct contributes to brain chemistry changes in specific brain regions
Study of mouse models provides clear implications for new targets to treat alcohol use disorder and fetal alcohol syndrome.

Scientists predict the areas of the brain to stimulate transitions between different brain states
Using a computer model of the brain, Gustavo Deco, director of the Center for Brain and Cognition, and Josephine Cruzat, a member of his team, together with a group of international collaborators, have developed an innovative method published in Proceedings of the National Academy of Sciences on Sept.

Read More: Brain News and Brain Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.