A Mind Reader? This Computer Understands What You Really Want

June 21, 1996

Lots of computers are smart enough these days to hear what you say and do what you ask: Voice recognition software for computers is a rapidly expanding field in computer programming.

But an experimental program developed at the University of Rochester makes the leap from "voice recognition" to "intention recognition," taking computers farther than ever before in the direction of understanding normal conversation. Scientists will unveil the program at next week's meeting of the Association for Computational Linguistics in Santa Cruz, Calif.

In the Trains Project, a computerized personal assistant named "Phenelope DuJour" helps a human engineer route trains through cities in the Northeast. The engineer and Phenelope discuss anything that could affect the best way to route a train, from heavy traffic, to flooding, to a visit from a presidential delegation.

"This is much more than speech recognition and synthesis," says James Allen, the John H. Dessauer Professor of Computer Science, whose system uses a commercial speech synthesis system and a speech recognition system provided by Carnegie Mellon University for those functions. "Speech recognition, while important, is only the beginning. The critical missing component to a conversational computer has been understanding and reasoning." Presenting the Trains Project with Allen are graduate students Eric Ringger and Teresa Sikorski and programmer Brad Miller.

Up until now, voice-recognition systems have required humans to adapt to computers, rather than the other way around. "Many speech recognition systems work pretty well if you stay within the constraints of the system," says Allen, "but we want a computer to do more than just pose a set of questions. Let's say you want to book an airplane trip by computer. Callers should be able to say, 'I want to fly from Detroit to Washington on July 4, with my mother, and she wants an aisle seat, and we want to fly at night and get the cheapest fare.' This is more like we really communicate."

But the problem with most systems is that many ordinary sentences are confusing to a computer. "My favorite is: 'Do you have the time?'" says Allen. "You could be asking what time it is. It could be a yes-or-no question. Or it can be intended as a gentle reminder that it's getting late."

Allen's team has taught Phenelope to deal with such ambiguous phrases. If the human manager says, "Take me to Chicago," for instance, or "New York is slow -- let's try Philadelphia," Phenelope usually interprets the sentence correctly or asks the speaker to clarify, enabling her to carry on a dialog where other computer systems falter.

While no one is predicting the demise of pop-up menus, icons, the mouse, or other computer-centered forms of communication, the work does show that it's possible to create a conversational computer that can be used by novices.

The team is now working on the next version of Trains, where the assistant will work in a more complex world of trains, planes and trucks that will shuttle boxcars and cargo between cities. The project, to which research associate George Ferguson has also contributed, is funded by the National Science Foundation, the Navy, and the Advanced Research Projects Agency.

Allen has wanted to be able to just sit down and talk to a computer ever since he was a graduate student at the University of Toronto 20 years ago. "That's the dream: To communicate with a machine as effortlessly as you do with another person." Allen is the author of Natural Language Understanding, the leading textbook for graduate students in the field.

University of Rochester

Related Speech Recognition Articles from Brightsurf:

How speech propels pathogens
Speech and singing spread saliva droplets, a phenomenon that has attracted much attention in the current context of the Covid-19 pandemic.

How everyday speech could transmit viral droplets
High-speed imaging of an individual producing common speech sounds shows that the sudden burst of airflow produced from the articulation of consonants like /p/ or /b/ carry salivary and mucus droplets for at least a meter in front of a speaker.

Speech processing hierarchy in the dog brain
Dog brains, just as human brains, process speech hierarchically: intonations at lower, word meanings at higher stages, according to a new study by Hungarian researchers.

Computational model decodes speech by predicting it
UNIGE scientists developed a neuro-computer model which helps explain how the brain identifies syllables in natural speech.

How the brain controls our speech
Speaking requires both sides of the brain. Each hemisphere takes over a part of the complex task of forming sounds, modulating the voice and monitoring what has been said.

How important is speech in transmitting coronavirus?
Normal speech by individuals who are asymptomatic but infected with coronavirus may produce enough aerosolized particles to transmit the infection, according to aerosol scientists at UC Davis.

Using a cappella to explain speech and music specialization
Speech and music are two fundamentally human activities that are decoded in different brain hemispheres.

Speech could be older than we thought
The theory of the 'descended larynx' has stated that before speech can emerge, the larynx must be in a low position to produce differentiated vowels.

How the brain detects the rhythms of speech
Neuroscientists at UC San Francisco have discovered how the listening brain scans speech to break it down into syllables.

Sound deprivation in one ear leads to speech recognition difficulties
Chronic conductive hearing loss, which can result from middle-ear infections, has been linked to speech recognition deficits, according to the results of a new study of 240 patients, led by scientists at Massachusetts Eye and Ear.

Read More: Speech Recognition News and Speech Recognition Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.