Crime fighting potential for computerised lip-reading

February 21, 2007

Researchers at the University of East Anglia are about to embark on an innovative new project to develop computer lip-reading systems that could be used for fighting crime.

The three-year project, which starts next month, will collect data for lip-reading and use it to create machines that automatically convert videos of lip-motions into text. It builds on work already carried out at UEA to develop state-of-the-art speech reading systems.

The university is teaming up with the Centre for Vision, Speech & Signal Processing at Surrey University, who have built accurate and reliable face and lip trackers, and the Home Office Scientific Development Branch, who want to investigate the feasibility of using the technology for crime fighting.

The team also hope to carry out computerised lip-reading of other languages.

While it is known that humans can and do lip-read, not much is known about exactly what visual information is needed for effective lip-reading. Human lip-reading can be unreliable, even using trained lip-readers.

Dr Richard Harvey, senior lecturer at UEA's School of Computing Sciences, is leading the project, which has been awarded £391,814 by the Engineering and Physical Sciences Research Council.

"We all lip read, for example in noisy situations like a bar or party, but even the performance of expert lip readers can be very poor," he said. "It appears that the best lip-readers are the ones who learned to speak a language before they lost their hearing and who have been taught lip-reading intensively. It is a very desirable skill."

Dr Harvey added: "The Home Office Scientific Development Branch is interested in anything that helps the police gather information about criminals or gather evidence."

As well as crime fighting there could be other potential uses for the technology, such as installing a camera in a mobile phone, or on the dash board for in-car speech recognition systems.

Another reason for developing computerised lip-reading is that the number of trained lip-readers is falling, mainly because people tend to be taught to sign instead.

Dr Harvey said: "To be effective the systems must accurately track the head over a variety of poses, extract numbers, or features, that describe the lips and then learn what features correspond to what text.

"To tackle the problem we will need to use information collected from audio speech. So this project will also investigate how to use the extensive information known about audio speech to recognise visual speech.

"The work will be highly experimental. We hope to have produced a system that will demonstrate the ability to lip-read in more general situations than we have done so far."
-end-


University of East Anglia

Related Mobile Phone Articles from Brightsurf:

Swirl power: how gentle body movement will charge your mobile phone
Scientists have discovered a way to generate electricity from nylon - the stretchy fabric used widely in sportswear and other shape-hugging apparel - raising hopes that the clothes on our backs will become an important source of energy.

Association of mobile phone location data indications of travel, stay-at-home mandates with COVID-19 infection rates in US
Anonymous mobile phone location data were used to examine travel and home dwelling time patterns before and after enactment of stay-at-home orders in US states to examine associations between changes in mobility and the COVID-19 curve.

Security gap allows eavesdropping on mobile phone calls
Calls via the LTE mobile network, also known as 4G, are encrypted and should therefore be tap-proof.

Some mobile phone apps may contain hidden behaviors that users never see
A team of cybersecurity researchers has discovered that a large number of cell phone applications contain hardcoded secrets allowing others to access private data or block content provided by users.

How secure are four and six-digit mobile phone PINs?
A German-American team of IT security researchers has investigated how users choose the PIN for their mobile phones and how they can be convinced to use a more secure number combination.

World's smelliest fruit could charge your mobile phone
Pungent produce packs an electrical punch. New method using world's 'most repulsive smelling fruit' could 'substantially reduce' the cost of energy storage.

LTE vulnerability: Attackers can impersonate other mobile phone users
Exploiting a vulnerability in the mobile communication standard LTE, also known as 4G, researchers at Ruhr-Universit├Ąt Bochum can impersonate mobile phone users.

A photo taken with a mobile phone to detect frauds in rice labelling
Including plastic that is undetectable by the consumer or distorting the quality of the product are some of the frauds facing the third most consumed cereal in the world: rice.

Mapping disease outbreaks in urban settings using mobile phone data
A new EPFL and MIT study into the interplay between mobility and the 2013 and 2014 dengue outbreaks in Singapore has uncovered a legal void around access to mobile phone data -- information that can prove vital in preventing the spread of infectious diseases.

Mobile phone data reveals non-market value of coastal tourism under climate change
Big data application is an emerging field in climate change adaptation.

Read More: Mobile Phone News and Mobile Phone Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.