Single click generates lists to end all lists

May 05, 2004

USING search engines to compile a list- like the top 50 greatest blues guitarists by record sales, say- involves a lot of drudge work because you have to visit many web pages to gather the data you need. But the next step in search engine technology could make creating such lists possible with a single mouse click. KnowItAll, a search engine under development at the University of Washington, Seattle, trawls the web for data and then collates it in the form of a list.

The approach is unique, says its developer, Oren Etzioni, because it generates information that probably doesn't exist on any single web page. The US Department of Defense's research arm, DARPA, and Google, are so impressed that they are providing funding for the project.

Etzioni's ultimate aim is to have KnowItAll answer questions such as "list all British scientists born before 1900". The software cannot do that yet, because it lacks a module that can understand "natural-language" questions of this type. That will come later, he says. What it can do, however, is take a phrase like "list scientists" and return with a list that it believes with a high degree of confidence are (or were) scientists.

For any input noun- "scientists", "guitarists", "gardeners" or "actors", say- KnowItAll tries to find sentences on websites that contain that noun and looks for words that often appear after it. In this way it might find the phrases "scientists such as" and "scientists including". It then feeds these to 12 search engines and extracts the words that tend to follow, which are often scientists' names. But c1ertain phrases like "scientists such as botanists" also fulfil the search criteria. The software can work out that "botanists" is not a name, and it can use this to inject "botanists such as" into the engines to obtain an even fuller list of scientists' names.

KnowItAll then returns a long list of scientists' names- each one accompanied by its percentage probability of being correct, as measured by frequency of occurrence of the names on websites. Users will be able to choose the level of confidence they want in the data. KnowItAll is also able to find words that often occur close to the search term. In the case of "scientists" these might be words like "DNA" and "quantum". It uses them to refine the probability that a person is indeed a scientist.
Author: Celeste Biever

New Scientist issue: 8 May 2004


"These articles are posted on this site to give advance access to other authorised media who may wish to quote extracts as part of fair dealing with this copyrighted material. Full attribution is required, and if publishing online a link to is also required. Advance permission is required before any and every reproduction of each article in full - please contact Please note that all material is copyright of Reed Business Information Limited and we reserve the right to take such action as we consider appropriate to protect such copyright."

New Scientist

Related Search Engine Articles from Brightsurf:

Student research team develops hybrid rocket engine
In a year defined by obstacles, a University of Illinois at Urbana-Champaign student rocket team persevered.

Deep learning: A new engine for ecological resource research
Deep learning is driven by big data, which brings new opportunities for target classification, detection, semantic segmentation, instance segmentation, and regression in ecological resource research.

Microbiome search engine can increase efficiency in disease detection and diagnosis
An international team of researchers has proposed a microbiome search-based method, via Microbiome Search Engine, to analyze the wealth of available health data to detect and diagnose human diseases.  

Researchers wake monkeys by stimulating 'engine' of consciousness in brain
A small amount of electricity delivered at a specific frequency to a particular point in the brain will snap a monkey out of even deep anesthesia, pointing to a circuit of brain activity key to consciousness and suggesting potential treatments for debilitating brain disorders.

Knowledge Engine is ready to accelerate genomic research
Five years ago, a team of computer scientists, biomedical researchers, and bioinformaticians set out to bring the power of collective knowledge to genomic research.

Locomotor engine in the spinal cord revealed
Researchers at Karolinska Institutet in Sweden have revealed a new principle of organization which explains how locomotion is coordinated in vertebrates akin to an engine with three gears.

Physicists create world's smallest engine
The research explains how random fluctuations affect the operation of microscopic machines like this tiny motor.

The web meets genomics: a DNA search engine for microbes
Microbes are the most common and diverse organisms on the planet.

Scientists develop microbiome search engine to assess microbiome novelty and impact
Scientists from the Qingdao Institute of Bioenergy and Bioprocess Technology developed a way to objectively evaluate the novelty and impact of plethora of microbiomes in the vast universe of microbiome big-data, based on an innovative tool called Microbiome Search Engine (MSE).

A wrench in Earth's engine
Researchers from CU Boulder report that they may have pinned down the cause of 'stagnant slabs,' which resemble a wrench in the engine of the planet.

Read More: Search Engine News and Search Engine Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to