Nav: Home

Study reveals brain activity patterns underlying fluent speech

June 01, 2018

When we speak, we engage nearly 100 muscles, continuously moving our lips, jaw, tongue, and throat to shape our breath into the fluent sequences of sounds that form our words and sentences. A new study by UC San Francisco scientists reveals how these complex articulatory movements are coordinated in the brain.

The new research reveals that the brain's speech centers are organized more according to the physical needs of the vocal tract as it produces speech than by how the speech sounds (its "phonetics"). Linguists divide speech into abstract units of sound called "phonemes" and consider the /k/ sound in "keep" the same as the /k/ in "coop." But in reality, your mouth forms the sound differently in these two words to prepare for the different vowels that follow, and this physical distinction now appears to be more important to the brain regions responsible for producing speech than the theoretical sameness of the phoneme.

The findings, which extend previous studies on how the brain interprets the sounds of spoken language, could help guide the creation of new generation of prosthetic devices for those who are unable to speak: brain implants could monitor neural activity related to speech production and rapidly and directly translate those signals into synthetic spoken language.

The new study, published on May 17, 2018, in Neuron, was conducted by Josh Chartier and Gopala K. Anumanchipalli, PhD, both researchers in the laboratory of senior author Edward Chang, MD, professor of neurological surgery, Bowes Biomedical Investigator, and member of the UCSF Weill Institute for Neurosciences. They were joined by Keith Johnson, PhD, professor of linguistics at UC Berkeley.

A neural code for vocal tract movements

Chang, a neurosurgeon at the UCSF Epilepsy Center, specializes in surgeries to remove brain tissue that causes seizures in patients with epilepsy. In some cases, to prepare for these operations, he places high-density arrays of tiny electrodes onto the surface of the patients' brains, both to help identify the location triggering the patients' seizures and to map out other important areas, such as those involved in language, to make sure the surgery avoids damaging them.

In addition to its clinical importance, this method, known as electrocorticography, or ECoG, is a powerful tool for research. "It's a unique means of looking at thousands of neurons activating in unison," Chartier said.

In the new study, Chartier and Anumanchipalli asked five volunteers awaiting surgery, with ECoG electrodes placed over a region of ventral sensorimotor cortex that is a key center of speech production, to read aloud a collection of 460 natural sentences. The sentences were expressly constructed to encapsulate nearly all the possible articulatory contexts in American English. This comprehensiveness was crucial to capture the complete range of "coarticulation," the blending of phonemes that is essential to natural speech.

"Without coarticulation, our speech would be blocky and segmented to the point where we couldn't really understand it," said Chartier.

The research team was not able to simultaneously record the volunteers' neural activity and their tongue, mouth and larynx movements. Instead, they recorded only audio of the volunteers speaking and developed a novel deep learning algorithm to estimate which movements were made during specific speaking tasks.

This approach allowed the researchers to identify distinct populations of neurons responsible for the specific vocal tract movement patterns needed to produce fluent speech sounds, a level of complexity that had not been seen in previous experiments that used simpler syllable-by-syllable speech tasks.

The experiments revealed that a remarkable diversity of different movements were encoded by neurons surrounding individual electrodes. The researchers found there were four emergent groups of neurons that appeared to be responsible for coordinating movements of muscles of the lips, tongue, and throat into the four main configurations of the vocal tract used in American English. The researchers also identified neural populations associated with specific classes of phonetic phenomena, including separate clusters for consonants and vowels of different types, but their analysis suggested that these phonetic groupings were more of a byproduct of more natural groupings based on different types of muscle movement.

Regarding coarticulation, the researchers discovered that our brains' speech centers coordinate different muscle movement patterns based on the context of what's being said, and the order in which different sounds occur. For example, the jaw opens more to say the word "tap" than to say the word "has" -- despite having the same vowel sound (/ae/), the mouth has to get ready to close to make the /z/ sound in "has." The researchers found that neurons in the ventral sensorimotor cortex were highly attuned to this and other co-articulatory features of English, suggesting that the brain cells are tuned to produce fluid, context-dependent speech as opposed to reading out discrete speech segments in serial order.

"During speech production, there is clearly another layer of neural processing that happens, which enables the speaker to merge phonemes together into something the listener can understand," said Anumanchipalli.

Path to a Speech Prosthetic

"This study highlights why we need to take into account vocal tract movements and not just linguistic features like phonemes when studying speech production," Chartier said. He thinks that this work paves the way not only for additional studies that tackle the sensorimotor aspect of speech production, but could also pay practical dividends.

"We know now that the sensorimotor cortex encodes vocal tract movements, so we can use that knowledge to decode cortical activity and translate that via a speech prosthetic," said Chartier. "This would give voice to people who can't speak but have intact neural functions."

Ultimately, the study could represent a new research avenue for Chartier and Anumanchipalli's team at UCSF. "It's really made me think twice about how phonemes fit in--in a sense, these units of speech that we pin so much of our research on are just byproducts of a sensorimotor signal," Anumanchipalli said.

This work was supported by grants from the NIH (DP2 OD008627 and U01 NS098971-01). E.F.C. is also a New York Stem Cell Foundation-Robertson Investigator. This research was also supported by the New York Stem Cell Foundation, the Howard Hughes Medical Institute, the McKnight Foundation, the Shurl and Kay Curci Foundation, and the William K. Bowes Foundation.

About UCSF: UC San Francisco (UCSF) is a leading university dedicated to promoting health worldwide through advanced biomedical research, graduate-level education in the life sciences and health professions, and excellence in patient care. It includes top-ranked graduate schools of dentistry, medicine, nursing and pharmacy; a graduate division with nationally renowned programs in basic, biomedical, translational and population sciences; and a preeminent biomedical research enterprise. It also includes UCSF Health, which comprises top-ranked hospitals, UCSF Medical Center and UCSF Benioff Children's Hospitals in San Francisco and Oakland - and other partner and affiliated hospitals and healthcare providers throughout the Bay Area. Please visit http://www.ucsf.edu/news.
-end-


University of California - San Francisco

Related Neurons Articles:

A molecule that directs neurons
A research team coordinated by the University of Trento studied a mass of brain cells, the habenula, linked to disorders like autism, schizophrenia and depression.
Shaping the social networks of neurons
Identification of a protein complex that attracts or repels nerve cells during development.
With these neurons, extinguishing fear is its own reward
The same neurons responsible for encoding reward also form new memories to suppress fearful ones, according to new research by scientists at The Picower Institute for Learning and Memory at MIT.
How do we get so many different types of neurons in our brain?
SMU (Southern Methodist University) researchers have discovered another layer of complexity in gene expression, which could help explain how we're able to have so many billions of neurons in our brain.
These neurons affect how much you do, or don't, want to eat
University of Arizona researchers have identified a network of neurons that coordinate with other brain regions to influence eating behaviors.
Mood neurons mature during adolescence
Researchers have discovered a mysterious group of neurons in the amygdala -- a key center for emotional processing in the brain -- that stay in an immature, prenatal developmental state throughout childhood.
Connecting neurons in the brain
Leuven researchers uncover new mechanisms of brain development that determine when, where and how strongly distinct brain cells interconnect.
The salt-craving neurons
Pass the potato chips, please! New research discovers neural circuits that regulate craving and satiation for salty tastes.
When neurons are out of shape, antidepressants may not work
Selective serotonin reuptake inhibitors (SSRIs) are the most commonly prescribed medication for major depressive disorder (MDD), yet scientists still do not understand why the treatment does not work in nearly thirty percent of patients with MDD.
Losing neurons can sometimes not be that bad
Current thinking about Alzheimer's disease is that neuronal cell death in the brain is to blame for the cognitive havoc caused by the disease.
More Neurons News and Neurons Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Reinvention
Change is hard, but it's also an opportunity to discover and reimagine what you thought you knew. From our economy, to music, to even ourselves–this hour TED speakers explore the power of reinvention. Guests include OK Go lead singer Damian Kulash Jr., former college gymnastics coach Valorie Kondos Field, Stockton Mayor Michael Tubbs, and entrepreneur Nick Hanauer.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Dispatch 6: Strange Times
Covid has disrupted the most basic routines of our days and nights. But in the middle of a conversation about how to fight the virus, we find a place impervious to the stalled plans and frenetic demands of the outside world. It's a very different kind of front line, where urgent work means moving slow, and time is marked out in tiny pre-planned steps. Then, on a walk through the woods, we consider how the tempo of our lives affects our minds and discover how the beats of biology shape our bodies. This episode was produced with help from Molly Webster and Tracie Hunte. Support Radiolab today at Radiolab.org/donate.