Nav: Home

Hearing through your fingers: Device that converts speech

June 03, 2019

Amsterdam, NL, June 3, 2019 - A novel study published in Restorative Neurology and Neuroscience provides the first evidence that a simple and inexpensive non-invasive speech-to-touch sensory substitution device has the potential to improve hearing in hearing-impaired cochlear implant patients, as well as individuals with normal hearing, to better discern speech in various situations like learning a second language or trying to deal with the "cocktail party effect." The device can provide immediate multisensory enhancement without any training.

"Despite recent advancements in hearing aid and cochlear implants, the most widespread surgical implant to restore audition, hearing-impaired users still encounter significant practical and social challenges with or without aids," explained lead investigator Amir Amedi, PhD, Department of Medical Neurobiology, Institute for Medical Research Israel-Canada, Faculty of Medicine, Hebrew University of Jerusalem, Hadassah Ein-Kerem, and The Cognitive Science Program, The Hebrew University of Jerusalem, Jerusalem, Israel. "In particular, they all struggle with understanding speech in challenging, noisy acoustic environments, especially in presence of a competing speaker."

The number of sensory deprived patients and auditory deprived patients is expected to rise so it is crucial to develop efficient techniques for auditory recovery designed to convey the missing information to patients. "We live in a world that is becoming steadily more multisensory and we really need to understand the mechanisms underlying multisensory perception and integration. Providing relevant information using the sense of touch can significantly help hearing," commented Dr. Amedi.

In this current proof-of-concept study investigators hypothesized that they would be able to improve speech understanding under challenging conditions by exploiting the ability of the brain to integrate information coming simultaneously from different senses. They designed a minimalistic auditory-to-tactile sensory substitution device (SSD) that transforms low-frequency speech signals into tactile vibrations delivered on two fingertips. They asked a group of non-native English speakers to repeat a series of sentences, which was degraded by embedding them in speech-like noise.

As expected, when participants could rely only on audition, their understanding of such sentences was poor. Crucially, however, their sentence understanding significantly improved when they paired the degraded speech signal with complementary vibration delivered on the participants fingertips. The vibration conveyed a specific set of frequencies known as fundamental frequencies that characterize speech signals.

The reported improvement at the group level was 6 dB - a major difference considering that an increase of 10 dB represents a doubling of the perceived loudness. These results are especially relevant when compared to earlier SSD studies showing effects in behavior only after demanding cognitive training. "Our results carry important implications for further research, as well as possible clinical and practical solutions," noted co-author Tomasz Wolak, PhD Eng, Head of the Bioimaging Research Center, Institute of Physiology and Pathology of Hearing, World Hearing Center, Warsaw, Poland. "The ability to 'hear through one's fingers can significantly help hearing. Our approach suggests that multisensory stimulations providing the same type of information (in this case spoken language conveyed through touch in addition to hearing) should be processed in the same brain region (in this case spoken language centers), ultimately then predicting that multisensory stimulations (both sounds and touch) should enhance perception.

According to lead author Katarzyna Cieśla, PhD, World Hearing Center, Warsaw, and Hebrew University of Jerusalem, "The most compelling aspect of our study is the fact that learning to use such speech-to-touch sensory substitution device did not require any training. We believe it can also serve as an aid for the elderly population, which finds it challenging to follow extensive training regimes. This might be the first study showing such immediate relevant enhancement of a sensory substitution device and suggests the brain is much more multisensory than the common wisdom."

"This study is a major step forward to introduce multisensory plasticity of the brain as an innovative paradigm to maximize the potential of patients to compensate for their sensory loss," commented Bernhard Sabel, PhD, Editor-in-Chief of Restorative Neurology and Neuroscience.

Next the team plans to further improve the device and training regimes in order to reach the goal of 10 dB enhancement and to test for human brain mechanisms using an MRI-compatible version of the device in various populations (both hearing and hearing-impaired people).
-end-
ABOUT RESTORATIVE NEUROLOGY AND NEUROSCIENCE (RNN)

An interdisciplinary journal under the editorial leadership of Bernhard Sabel, PhD, Restorative Neurology and Neuroscience publishes papers relating the plasticity and response of the nervous system to accidental or experimental injuries and their interventions, transplantation, neurodegenerative disorders and experimental strategies to improve regeneration or functional recovery and rehabilitation. Experimental and clinical research papers adopting fresh conceptual approaches are encouraged. The overriding criteria for publication are novelty, significant experimental or clinical relevance and interest to a multidisciplinary audience.http://www.iospress.nl/journal/restorative-neurology-and-neuroscience

RNN Editorial Office
Prof. Dr. Bernhard Sabel
Institut für Medizinische Psychologie
Medizinische Fakultät
Otto-v.-Guericke Universität Magdeburg
39120 Magdeburg/Germany
Tel: +49-391-672-1800
Fax: +49-391-672-1803
E-mail: rnn@med.ovgu.de

ABOUT IOS PRESS

IOS Press is headquartered in Amsterdam with satellite offices in the USA, Germany, India and China and serves the information needs of scientific and medical communities worldwide. IOS Press now publishes more than 80 international peer-reviewed journals and about 75 book titles each year on subjects ranging from computer science, artificial intelligence, and engineering to medicine, neuroscience, and cancer research. http://www.iospress.com

IOS Press

Related Language Articles:

Chinese to rise as a global language
With the continuing rise of China as a global economic and trading power, there is no barrier to prevent Chinese from becoming a global language like English, according to Flinders University academic Dr Jeffrey Gil.
'She' goes missing from presidential language
MIT researchers have found that although a significant percentage of the American public believed the winner of the November 2016 presidential election would be a woman, people rarely used the pronoun 'she' when referring to the next president before the election.
How does language emerge?
How did the almost 6000 languages of the world come into being?
New research quantifies how much speakers' first language affects learning a new language
Linguistic research suggests that accents are strongly shaped by the speaker's first language they learned growing up.
Why the language-ready brain is so complex
In a review article published in Science, Peter Hagoort, professor of Cognitive Neuroscience at Radboud University and director of the Max Planck Institute for Psycholinguistics, argues for a new model of language, involving the interaction of multiple brain networks.
Do as i say: Translating language into movement
Researchers at Carnegie Mellon University have developed a computer model that can translate text describing physical movements directly into simple computer-generated animations, a first step toward someday generating movies directly from scripts.
Learning language
When it comes to learning a language, the left side of the brain has traditionally been considered the hub of language processing.
Learning a second alphabet for a first language
A part of the brain that maps letters to sounds can acquire a second, visually distinct alphabet for the same language, according to a study of English speakers published in eNeuro.
Sign language reveals the hidden logical structure, and limitations, of spoken language
Sign languages can help reveal hidden aspects of the logical structure of spoken language, but they also highlight its limitations because speech lacks the rich iconic resources that sign language uses on top of its sophisticated grammar.
Lying in a foreign language is easier
It is not easy to tell when someone is lying.
More Language News and Language Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Meditations on Loneliness
Original broadcast date: April 24, 2020. We're a social species now living in isolation. But loneliness was a problem well before this era of social distancing. This hour, TED speakers explore how we can live and make peace with loneliness. Guests on the show include author and illustrator Jonny Sun, psychologist Susan Pinker, architect Grace Kim, and writer Suleika Jaouad.
Now Playing: Science for the People

#565 The Great Wide Indoors
We're all spending a bit more time indoors this summer than we probably figured. But did you ever stop to think about why the places we live and work as designed the way they are? And how they could be designed better? We're talking with Emily Anthes about her new book "The Great Indoors: The Surprising Science of how Buildings Shape our Behavior, Health and Happiness".
Now Playing: Radiolab

The Third. A TED Talk.
Jad gives a TED talk about his life as a journalist and how Radiolab has evolved over the years. Here's how TED described it:How do you end a story? Host of Radiolab Jad Abumrad tells how his search for an answer led him home to the mountains of Tennessee, where he met an unexpected teacher: Dolly Parton.Jad Nicholas Abumrad is a Lebanese-American radio host, composer and producer. He is the founder of the syndicated public radio program Radiolab, which is broadcast on over 600 radio stations nationwide and is downloaded more than 120 million times a year as a podcast. He also created More Perfect, a podcast that tells the stories behind the Supreme Court's most famous decisions. And most recently, Dolly Parton's America, a nine-episode podcast exploring the life and times of the iconic country music star. Abumrad has received three Peabody Awards and was named a MacArthur Fellow in 2011.