Nav: Home

Read my lips: New technology spells out what's said when audio fails

March 24, 2016

New lip-reading technology developed at the University of East Anglia (UEA) could help in solving crimes and provide communication assistance for people with hearing and speech impairments.

The visual speech recognition technology, created by Dr Helen L. Bear and Prof Richard Harvey of UEA's School of Computing Sciences, can be applied "any place where the audio isn't good enough to determine what people are saying," Dr Bear said.

Dr Bear, whose findings will be presented at the International Conference on Acoustics, Speech and Signal Processing (ICASSP) in Shanghai on March 25, said unique problems with determining speech arise when sound isn't available - such as on CCTV footage - or if the audio is inadequate and there aren't clues to give the context of a conversation. The sounds '/p/,' '/b/,' and '/m/' all look similar on the lips, but now the machine lip-reading classification technology can differentiate between the sounds for a more accurate translation.

Dr Bear said: "We are still learning the science of visual speech and what it is people need to know to create a fool-proof recognition model for lip-reading, but this classification system improves upon previous lip-reading methods by using a novel training method for the classifiers.

"Potentially, a robust lip-reading system could be applied in a number of situations, from criminal investigations to entertainment. Lip-reading has been used to pinpoint words footballers have shouted in heated moments on the pitch, but is likely to be of most practical use in situations where are there are high levels of noise, such as in cars or aircraft cockpits.

"Crucially, whilst there are still improvements to be made, such a system could be adapted for use for a range of purposes - for example, for people with hearing or speech impairments. Alternatively, a good lip-reading machine could be part of an audio-visual recognition system."

Prof Harvey said: "Lip-reading is one of the most challenging problems in artificial intelligence so it's great to make progress on one of the trickier aspects, which is how to train machines to recognise the appearance and shape of human lips."

The research was part of a three-year project and was supported by the Engineering and Physical Sciences Research Council (EPSRC).

The paper, Decoding visemes: Improving machine lip-reading, will be published on March 25, 2016 in the Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing 2016.
-end-


University of East Anglia

Related Engineering Articles:

Engineering a new cancer detection tool
E. coli may have potentially harmful effects but scientists in Australia have discovered this bacterium produces a toxin which binds to an unusual sugar that is part of carbohydrate structures present on cells not usually produced by healthy cells.
Engineering heart valves for the many
The Wyss Institute for Biologically Inspired Engineering and the University of Zurich announced today a cross-institutional team effort to generate a functional heart valve replacement with the capacity for repair, regeneration, and growth.
Geosciences-inspired engineering
The Mackenzie Dike Swarm and the roughly 120 other known giant dike swarms located across the planet may also provide useful information about efficient extraction of oil and natural gas in today's modern world.
Engineering success
Academically strong, low-income would-be engineers get the boost they need to complete their undergraduate degrees.
HKU Engineering Professor Ron Hui named a Fellow by the UK Royal Academy of Engineering
Professor Ron Hui, Chair Professor of Power Electronics and Philip Wong Wilson Wong Professor of Electrical Engineering at the University of Hong Kong, has been named a Fellow by the Royal Academy of Engineering, UK, one of the most prestigious national academies.
Engineering a better biofuel
The often-maligned E. coli bacteria has powerhouse potential: in the lab, it has the ability to crank out fuels, pharmaceuticals and other useful products at a rapid rate.
Pascali honored for contributions to engineering education
Raresh Pascali, instructional associate professor in the Mechanical Engineering Technology Program at the University of Houston, has been named the 2016 recipient of the Ross Kastor Educator Award.
Scaling up tissue engineering
A team at the Wyss Institute for Biologically Inspired Engineering at Harvard University and the Harvard John A.
Engineering material magic
University of Utah engineers have discovered a new kind of 2-D semiconducting material for electronics that opens the door for much speedier computers and smartphones that also consume a lot less power.
Engineering academic elected a Fellow of the IEEE
A University of Bristol academic has been elected a Fellow of the world's largest and most prestigious professional association for the advancement of technology.

Related Engineering Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Anthropomorphic
Do animals grieve? Do they have language or consciousness? For a long time, scientists resisted the urge to look for human qualities in animals. This hour, TED speakers explore how that is changing. Guests include biological anthropologist Barbara King, dolphin researcher Denise Herzing, primatologist Frans de Waal, and ecologist Carl Safina.
Now Playing: Science for the People

#SB2 2019 Science Birthday Minisode: Mary Golda Ross
Our second annual Science Birthday is here, and this year we celebrate the wonderful Mary Golda Ross, born 9 August 1908. She died in 2008 at age 99, but left a lasting mark on the science of rocketry and space exploration as an early woman in engineering, and one of the first Native Americans in engineering. Join Rachelle and Bethany for this very special birthday minisode celebrating Mary and her achievements. Thanks to our Patreons who make this show possible! Read more about Mary G. Ross: Interview with Mary Ross on Lash Publications International, by Laurel Sheppard Meet Mary Golda...