Nav: Home

Read my lips: New technology spells out what's said when audio fails

March 24, 2016

New lip-reading technology developed at the University of East Anglia (UEA) could help in solving crimes and provide communication assistance for people with hearing and speech impairments.

The visual speech recognition technology, created by Dr Helen L. Bear and Prof Richard Harvey of UEA's School of Computing Sciences, can be applied "any place where the audio isn't good enough to determine what people are saying," Dr Bear said.

Dr Bear, whose findings will be presented at the International Conference on Acoustics, Speech and Signal Processing (ICASSP) in Shanghai on March 25, said unique problems with determining speech arise when sound isn't available - such as on CCTV footage - or if the audio is inadequate and there aren't clues to give the context of a conversation. The sounds '/p/,' '/b/,' and '/m/' all look similar on the lips, but now the machine lip-reading classification technology can differentiate between the sounds for a more accurate translation.

Dr Bear said: "We are still learning the science of visual speech and what it is people need to know to create a fool-proof recognition model for lip-reading, but this classification system improves upon previous lip-reading methods by using a novel training method for the classifiers.

"Potentially, a robust lip-reading system could be applied in a number of situations, from criminal investigations to entertainment. Lip-reading has been used to pinpoint words footballers have shouted in heated moments on the pitch, but is likely to be of most practical use in situations where are there are high levels of noise, such as in cars or aircraft cockpits.

"Crucially, whilst there are still improvements to be made, such a system could be adapted for use for a range of purposes - for example, for people with hearing or speech impairments. Alternatively, a good lip-reading machine could be part of an audio-visual recognition system."

Prof Harvey said: "Lip-reading is one of the most challenging problems in artificial intelligence so it's great to make progress on one of the trickier aspects, which is how to train machines to recognise the appearance and shape of human lips."

The research was part of a three-year project and was supported by the Engineering and Physical Sciences Research Council (EPSRC).

The paper, Decoding visemes: Improving machine lip-reading, will be published on March 25, 2016 in the Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing 2016.
-end-


University of East Anglia

Related Engineering Articles:

Engineering a new cancer detection tool
E. coli may have potentially harmful effects but scientists in Australia have discovered this bacterium produces a toxin which binds to an unusual sugar that is part of carbohydrate structures present on cells not usually produced by healthy cells.
Engineering heart valves for the many
The Wyss Institute for Biologically Inspired Engineering and the University of Zurich announced today a cross-institutional team effort to generate a functional heart valve replacement with the capacity for repair, regeneration, and growth.
Geosciences-inspired engineering
The Mackenzie Dike Swarm and the roughly 120 other known giant dike swarms located across the planet may also provide useful information about efficient extraction of oil and natural gas in today's modern world.
Engineering success
Academically strong, low-income would-be engineers get the boost they need to complete their undergraduate degrees.
HKU Engineering Professor Ron Hui named a Fellow by the UK Royal Academy of Engineering
Professor Ron Hui, Chair Professor of Power Electronics and Philip Wong Wilson Wong Professor of Electrical Engineering at the University of Hong Kong, has been named a Fellow by the Royal Academy of Engineering, UK, one of the most prestigious national academies.
More Engineering News and Engineering Current Events

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Teaching For Better Humans
More than test scores or good grades — what do kids need to prepare them for the future? This hour, guest host Manoush Zomorodi and TED speakers explore how to help children grow into better humans, in and out of the classroom. Guests include educators Olympia Della Flora and Liz Kleinrock, psychologist Thomas Curran, and writer Jacqueline Woodson.
Now Playing: Science for the People

#535 Superior
Apologies for the delay getting this week's episode out! A technical glitch slowed us down, but all is once again well. This week, we look at the often troubling intertwining of science and race: its long history, its ability to persist even during periods of disrepute, and the current forms it takes as it resurfaces, leveraging the internet and nationalism to buoy itself. We speak with Angela Saini, independent journalist and author of the new book "Superior: The Return of Race Science", about where race science went and how it's coming back.