Do reassuring computers give pilots a false sense of security?

November 02, 1999

COMPUTER systems designed to reduce human error in aircraft cockpits, nuclear plants and intensive care units can have the opposite effect. A new study is the first to make a direct comparison between performance on tasks carried out with and without the aid of a computer. Rather than improving overall performance and decision making, the researchers found that in certain situations the computer prompts actually made people more prone to errors.

Linda Skitka, a psychologist at the University of Illinois at Chicago, working with colleagues at San Francisco State University and NASA's Ames Research Center in Moffett Field, California, used a basic flight simulator to study 80 student volunteers carrying out flight tasks.

The volunteers were trained to use the simulator and given two practice sessions lasting five minutes. Then each had to fly eight "missions" involving a total of 100 tasks, such as pressing buttons when they passed beacons or when gauge levels dropped. Half the students were given automated cues-which they were told would be almost perfect, though not infallible-to remind them to carry out the tasks; the machine gave a false cue six times and failed to notify them of six events. The other volunteers just used instrument readings on the simulator's display, which both groups had been told would be 100 per cent accurate.

Volunteers using automated cues on average carried out only 59 per cent of the tasks that the computer hadn't prompted them to do. But those who relied on the instrument readings were 97 per cent accurate in performing these tasks. The computer-prompted volunteers also carried out 65 per cent of the wrongly prompted tasks, despite contradictory instrument readings.

Overall, this meant that those prompted by the computer performed no better than those who relied on instruments alone. Skitka believes this reduction in vigilance stems from the human tendency to be a "cognitive miser". People delegate tasks that they think don't require their full attention-in this case to the computer.

Computer systems used in safety-critical situations should be more reliable than Skitka's. Pilots and nuclear plant operators are also more highly trained. But given that any mistake can be fatal, Skitka says there's no room for complacency. "I think there is some cause for concern, given that automated decision aids are often used where even one error can be catastrophic."

As an example, Skitka cites the Korean Airlines Flight 007 disaster. This airliner was shot down by Soviet fighters in 1983 for violating restricted airspace, with the loss of all 269 on board, after its crew apparently relied on an automated flight path that had been set incorrectly. Lack of vigilance meant that the crew failed to notice where they were going as they veered off course (see Map).

Flight management computers are assuming ever greater control in civil aviation. Given Skitka's results, that's a disturbing trend. She believes the solution isn't to remove the technology, but to restrict the use of computer prompts to situations in which errors are likely to be most dangerous. Previous research has suggested that continual prompting can add to the risk that people will lose concentration.

Convincing pilots to abandon their computer helpers may not be easy. "Pilots like new technology," says John Mazor, spokesman for the Air Line Pilots Association International in Washington DC. However, he says, the correct balance of control between the pilot and automated systems is an ongoing debate.
Author: Duncan Graham-Rowe

Source: International Journal of Human-Computer Studies (vol 51, p 991)

New Scientist issue 6th November 99


New Scientist

Related Computer Articles from Brightsurf:

UCLA computer scientists set benchmarks to optimize quantum computer performance
Two UCLA computer scientists have shown that existing compilers, which tell quantum computers how to use their circuits to execute quantum programs, inhibit the computers' ability to achieve optimal performance.

Digitize your dog into a computer game
Researchers from CAMERA at the University of Bath have developed motion capture technology that enables you to digitise your dog without a motion capture suit and using only one camera.

Stabilizing brain-computer interfaces
Researchers from Carnegie Mellon University (CMU) and the University of Pittsburgh (Pitt) have published research in Nature Biomedical Engineering that will drastically improve brain-computer interfaces and their ability to remain stabilized during use, greatly reducing or potentially eliminating the need to recalibrate these devices during or between experiments.

Computer-generated genomes
Professor Beat Christen, ETH Zurich to speak in the AAAS 2020 session, 'Synthetic Biology: Digital Design of Living Systems.' Christen will describe how computational algorithms paired with chemical DNA synthesis enable digital manufacturing of biological systems up to the size of entire microbial genomes.

Computer-based weather forecast: New algorithm outperforms mainframe computer systems
The exponential growth in computer processing power seen over the past 60 years may soon come to a halt.

A computer that understands how you feel
Neuroscientists have developed a brain-inspired computer system that can look at an image and determine what emotion it evokes in people.

Computer program looks five minutes into the future
Scientists from the University of Bonn have developed software that can look minutes into the future: The program learns the typical sequence of actions, such as cooking, from video sequences.

Computer redesigns enzyme
University of Groningen biotechnologists used a computational method to redesign aspartase and convert it to a catalyst for asymmetric hydroamination reactions.

Mining for gold with a computer
Engineers from Texas A&M University and Virginia Tech report important new insights into nanoporous gold -- a material with growing applications in several areas, including energy storage and biomedical devices -- all without stepping into a lab.

Teaching quantum physics to a computer
An international collaboration led by ETH physicists has used machine learning to teach a computer how to predict the outcomes of quantum experiments.

Read More: Computer News and Computer Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to