Nav: Home

Brain patterns can predict speech of words and syllables

December 10, 2019

Neurons in the brain's motor cortex previously thought of as active mainly during hand and arm movements also light up during speech in a way that is similar to patterns of brain activity linked to these movements, suggest new findings published today in eLife.

By demonstrating that it is possible to identify different syllables or words from patterns of neural activity, the study provides insights that could potentially be used to restore the voice in people who have lost the ability to speak.

Speaking involves some of the most precise and coordinated movements humans make. Studying it is fascinating but challenging, because there are few opportunities to make measurements from inside someone's brain while they speak. This study took place as part of the BrainGate2 Brain-Computer Interface pilot clinical trial*, which is testing a computer device that can 'communicate' with the brain, helping to restore communication and provide control of prosthetics such as robotic arms.

The researchers studied speech by recording brain activity from multi-electrode arrays previously placed in the motor cortex of two people taking part in BrainGate 2 study. This allowed them to study the timing and location of the firing of a large population of neurons that is activated during speech, rather than just a few at a time.

"We first asked if neurons in the so-called 'hand knob' area of the brain's motor cortex are active during speaking," explains lead author Sergey Stavisky, Postdoctoral Research Fellow in the Department of Neurosurgery and the Wu Tsai Neurosciences Institute at Stanford University, US. "This seemed unlikely because this is an area known to control hand and arm movements, not speech. But clues in the scientific literature suggested there might be an overlap."

To test this, the team recorded neural activity from participants during a speaking task where they heard one of 10 different syllables, or one of 10 different short words, and then spoke the prompted sound after hearing a 'go' cue. The firing rates of neurons changed strongly during their speaking of words and syllables and the active neurons were spread throughout the part of motor cortex that the researchers were recording from. Moreover, the firing rates did not change as much when the participants heard the sound, only when they spoke it. This change in neuron activity also corresponded to groups of similar sounds, called phonemes, and with face and mouth movements. This suggests that although there is high-level separation of control of different body parts across the motor cortex, activity of the individual neurons overlaps.

Next, the team performed a 'decoding' analysis to see whether the neuron-firing patterns could reveal information about the specific sound being spoken. They found that by analysing neural activity across nearly 200 electrodes, they could identify which of several syllables or words the participant was saying. In fact, certain patterns of neuron activity could correctly predict the sound, or lack of sound, in more than 80% of cases for one of the participants, and between 55% and 61% of cases for the other.

"This suggests it might be possible to use this brain activity to understand what words people who cannot speak are trying to say," says co-senior author Krishna Shenoy, Howard Hughes Medical Institute Investigator and Hong Seh and Vivian W. M. Lim Professor in the School of Engineering, and Co-Director of the Neural Prosthetics Translational Laboratory (NPTL), at Stanford University.

"With this study we have shown that we can identify syllables and words people say based on their brain activity, which lays the groundwork for synthesising, or producing, text and speech from the neural activity measured when a patient tries to speak," concludes co-senior author Jaimie Henderson, John and Jene Blume - Robert and Ruth Halperin Professor in the Department of Neurosurgery and Co-Director of NPTL, Stanford University. "Further work is now needed to synthesise continuous speech for people who cannot provide example data by actually speaking."
-end-
References

The paper 'Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis' can be freely accessed online at https://doi.org/10.7554/eLife.46015. Contents, including text, figures and data, are free to reuse under a CC BY 4.0 license.

*More information about the BrainGate2 study is available at https://www.braingate.org.

Media contacts

Emily Packer, Senior Press Officer
eLife
e.packer@elifesciences.org
+44 1223 855373

Bruce Goldman, Science Writer
Stanford Health Care | School of Medicine
goldmanb@stanford.edu
+1 650 725-2106

About eLife

eLife is a non-profit organisation inspired by research funders and led by scientists. Our mission is to help scientists accelerate discovery by operating a platform for research communication that encourages and recognises the most responsible behaviours in science. We publish important research in all areas of the life and biomedical sciences, including Human Biology and Medicine, and Neuroscience, which is selected and evaluated by working scientists and made freely available online without delay. eLife also invests in innovation through open-source tool development to accelerate research communication and discovery. Our work is guided by the communities we serve. eLife is supported by the Howard Hughes Medical Institute, the Max Planck Society, the Wellcome Trust and the Knut and Alice Wallenberg Foundation. Learn more at https://elifesciences.org/about.

To read the latest Human Biology and Medicine research published in eLife, visit https://elifesciences.org/subjects/human-biology-medicine.

And for the latest in Neuroscience, see https://elifesciences.org/subjects/neuroscience.

eLife

Related Neurons Articles:

The first 3D map of the heart's neurons
An interdisciplinary research team establishes a new technological pipeline to build a 3D map of the neurons in the heart, revealing foundational insight into their role in heart attacks and other cardiac conditions.
Mapping the neurons of the rat heart in 3D
A team of researchers has developed a virtual 3D heart, digitally showcasing the heart's unique network of neurons for the first time.
How to put neurons into cages
Football-shaped microscale cages have been created using special laser technologies.
A molecule that directs neurons
A research team coordinated by the University of Trento studied a mass of brain cells, the habenula, linked to disorders like autism, schizophrenia and depression.
Shaping the social networks of neurons
Identification of a protein complex that attracts or repels nerve cells during development.
With these neurons, extinguishing fear is its own reward
The same neurons responsible for encoding reward also form new memories to suppress fearful ones, according to new research by scientists at The Picower Institute for Learning and Memory at MIT.
How do we get so many different types of neurons in our brain?
SMU (Southern Methodist University) researchers have discovered another layer of complexity in gene expression, which could help explain how we're able to have so many billions of neurons in our brain.
These neurons affect how much you do, or don't, want to eat
University of Arizona researchers have identified a network of neurons that coordinate with other brain regions to influence eating behaviors.
Mood neurons mature during adolescence
Researchers have discovered a mysterious group of neurons in the amygdala -- a key center for emotional processing in the brain -- that stay in an immature, prenatal developmental state throughout childhood.
Connecting neurons in the brain
Leuven researchers uncover new mechanisms of brain development that determine when, where and how strongly distinct brain cells interconnect.
More Neurons News and Neurons Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

TED Radio Wow-er
School's out, but many kids–and their parents–are still stuck at home. Let's keep learning together. Special guest Guy Raz joins Manoush for an hour packed with TED science lessons for everyone.
Now Playing: Science for the People

#565 The Great Wide Indoors
We're all spending a bit more time indoors this summer than we probably figured. But did you ever stop to think about why the places we live and work as designed the way they are? And how they could be designed better? We're talking with Emily Anthes about her new book "The Great Indoors: The Surprising Science of how Buildings Shape our Behavior, Health and Happiness".
Now Playing: Radiolab

The Third. A TED Talk.
Jad gives a TED talk about his life as a journalist and how Radiolab has evolved over the years. Here's how TED described it:How do you end a story? Host of Radiolab Jad Abumrad tells how his search for an answer led him home to the mountains of Tennessee, where he met an unexpected teacher: Dolly Parton.Jad Nicholas Abumrad is a Lebanese-American radio host, composer and producer. He is the founder of the syndicated public radio program Radiolab, which is broadcast on over 600 radio stations nationwide and is downloaded more than 120 million times a year as a podcast. He also created More Perfect, a podcast that tells the stories behind the Supreme Court's most famous decisions. And most recently, Dolly Parton's America, a nine-episode podcast exploring the life and times of the iconic country music star. Abumrad has received three Peabody Awards and was named a MacArthur Fellow in 2011.