Nav: Home

Human brain tunes into visual rhythms in sign language

June 08, 2017

The human brain works in rhythms and cycles. These patterns occur at predictable frequencies that depend on what a person is doing and on what part of the brain is active during the behavior.

Similarly, there are rhythms and patterns out in the world, and for the last 20 years, scientists have been perplexed by the brain's ability to "entrain," or match up, with these patterns. Language is one of those areas in which scientists observe neural entrainment: When people listen to speech, their brain waves lock up with the volume-based rhythms they hear. Since people can't pay attention to everything happening in their environment at once, this phase locking is thought to help anticipate when important information is likely to appear.

Many studies have documented this phenomenon in language processing; however, it has been difficult to tell whether neural entrainment is specialized for spoken language. In a new study in the Proceedings of the National Academy of Sciences, University of Chicago scholars designed an experiment using sign language to answer that question.

"To determine if neural entrainment to language is specialized for speech or if it is a general-purpose tool that humans can use for anything that is temporally predictable, we had to go outside of speech and outside of auditory perception," said Geoffrey Brookshire, the study's lead author and a PhD student in the Department of Psychology.

Brookshire worked with Daniel Casasanto, assistant professor of psychology and leader of the Experience and Cognition Lab; Susan Goldin-Meadow, the Beardsley Ruml Distinguished Service Professor of in the Department of Psychology and an acclaimed scholar of language and gesture; Howard Nusbaum, professor of psychology and an expert in spoken language and language use; and Jenny Lu, a PhD student specializing in sign language, gesture and language development.

"By looking at sign, we're learning something about how the brain processes language more generally. We're solving a mystery we couldn't crack by studying speech alone," Casasanto said.

In speech, the brain locks on to syllables, words and phrases, and those rhythms occur below 8 Hz, or 8 pulses per second. Vision also has a preferred frequency onto which it latches.

"When we focus on random flashes of light, for example, our brains most enthusiastically lock on to flashes around 10 Hz. By looking at sign language, we can ask whether the important thing for entrainment is which sense you're using, or the kind of information you're getting," Brookshire said.

To determine if people tune into visual rhythms in the same way they tune into the auditory rhythms of language, they showed videos of stories told in American Sign Language to fluent signers and measured brain activity as they watched. Once the researchers had these electroencephalogram readings, they needed a way to measure visual rhythms in sign language.

While there are well-established methods to measure rhythms in speech, there are no automatic, objective equivalents for the temporal structure of sign language. So the researchers created one.

They developed a new metric, called the instantaneous visual change, which summarizes the degree of change at each time period during signing. They ran experiment videos, the ones watched by participants, through their new algorithm to identify peaks and valleys in visual changes between frames. The largest peaks were associated with large, quick movements.

With this roadmap illustrating the magnitude of visual changes over time in the videos, Brookshire overlaid the participants' EEGs to see whether people entrain around the normal visual frequency of about 10 Hz, or at the lower frequencies of signs and phrases in sign language -- about 2 Hz.

Their discovery answers a fundamental question that has been lingering for years in research on speech entrainment: Is it specialized for auditory speech? The study reveals that the brain entrains depending on the information in the signal--not on the differences between seeing and hearing. Participants' brain waves locked into the specific frequencies of sign language, rather than locking into the higher frequency that vision tends to prefer.

"This is an exciting finding because scientists have been theorizing for years about how adaptable or flexible entrainment may be, but we were never sure if it was specific to auditory processing or if it was more general purpose," Brookshire said. "This study suggests that humans have the ability to follow perceptual rhythms and make temporal predictions in any of our senses."

In a broader sense, neuroscientists want to understand how the human brain creates and perceives language, and entrainment has emerged as an important mechanism. In revealing neural entrainment as a generalized strategy for improving sensitivity to informational peaks, this study takes significant steps toward advancing the understanding of human language and perception.

"The piece of the paper that I find particularly exciting is that it compares how signers and non-signers process American Sign Language stimuli," Goldin-Meadow said. "Although both groups showed the same level of entrainment in early visual regions, they displayed differences in frontal regions -- this finding sets the stage for us to identify aspects of neural entrainment that are linked to the physical properties of the visual signal compared to aspects that appear only with linguistic knowledge."
-end-


University of Chicago

Related Language Articles:

The world's most spoken language is...'Terpene'
If you're small, smells are a good way to stand out.
Study analyzes what 'a' and 'the' tell us about language acquisition
A study co-authored by an MIT professor suggests that experience is an important component of early-childhood language usage although it doesn't necessarily account for all of a child's language facility.
Why do people switch their language?
Due to increasing globalization, the linguistic landscape of our world is changing; many people give up use of one language in favor of another.
Discovering what shapes language diversity
A research team led by Colorado State University is the first to use a form of simulation modeling to study the processes that shape language diversity patterns.
'Speaking my language': Method helps prepare teachers of dual language learners
Researchers at Lehigh University, led by L. Brook Sawyer and Patricia H.
The brain watched during language learning
Researchers from Nijmegen, the Netherlands, have for the first time captured images of the brain during the initial hours and days of learning a new language.
'Now-or-never bottleneck' explains language acquisition
We are constantly bombarded with linguistic input, but our brains are unable to remember long strings of linguistic information.
The secret language of microbes
Social microbes often interact with each other preferentially, favoring those that share certain genes in common.
A programming language for living cells
New language lets MIT researchers design novel biological circuits.
Syntax is not unique to human language
Human communication is powered by rules for combining words to generate novel meanings.

Related Language Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Jumpstarting Creativity
Our greatest breakthroughs and triumphs have one thing in common: creativity. But how do you ignite it? And how do you rekindle it? This hour, TED speakers explore ideas on jumpstarting creativity. Guests include economist Tim Harford, producer Helen Marriage, artificial intelligence researcher Steve Engels, and behavioral scientist Marily Oppezzo.
Now Playing: Science for the People

#524 The Human Network
What does a network of humans look like and how does it work? How does information spread? How do decisions and opinions spread? What gets distorted as it moves through the network and why? This week we dig into the ins and outs of human networks with Matthew Jackson, Professor of Economics at Stanford University and author of the book "The Human Network: How Your Social Position Determines Your Power, Beliefs, and Behaviours".