New digital image retrieval approach reduces search time, improves accuracy

June 20, 2001

University Park, Pa. -- A new approach to automatically sorting, classifying and retrieving digital images -- based on the way people look at and understand pictures -- promises not only faster, more accurate image database searches but also better Web searches too, says a Penn State researcher. Dr. James Z. Wang, Penn State assistant professor of information sciences and technology who holds the PNC Technologies Career Development Professorship, developed the approach when he was a graduate student at Stanford University. He says the new approach has potential for application in biomedicine, crime prevention, the military, commerce, education, entertainment and Web image classification.

The new approach does not consider any information other than the image itself. Just as a person, shown a picture of a horse, can extract the features characteristic of horses and then identify other pictures that contain horses, so does the new computer-based approach. The new system retrieves relevant images from an image database or the Web on the basis of automatically derived image features or content.

Image retrieval techniques currently in commercial use mostly rely on keywords or descriptions. While this text-based approach can be accurate and efficient for limited databases of high value, for example, museum pictures, it can become prohibitively expensive to input, manually, descriptions of large-scale image databases such as astronomical observations. The new approach not only reduces the need for textual information but also can handle, quickly and efficiently, the approximately one billion images that can be found on the Internet.

Wang and colleagues have built an experimental image retrieval system, called SIMPLIcity, to validate and demonstrate their methods and have tested it on a database of about 200,000 general-purpose images and an archive of more than 70,000 pathology images. The SIMPLIcity approach performs better and faster than existing methods and can also be applied to the classification of on-line images and web sites. (To view a demonstration of SIMPLIcity go to Using the same approach they have also developed an image filtering system, called WIPE, that parents can use to protect their children from pornography on the Web. WIPE identifies and blocks objectionable images. WIPE takes only one second per picture versus other filters that require minutes and has an accuracy over 90 percent. (A demonstration of WIPE is also at Wang has detailed his content-based image retrieval (CBIR) approach in a new book, Integrated Region-Based Image Retrieval, published this month (June) by Kluwer Academic Publishers. The book details the design and implementation of the new content-based retrieval system and its application to a general picture library and a biomedical image database.

Wang notes that the capability of existing CBIR systems is essentially limited by the fact that they rely on only primitive features of the image. In his new approach, Wang matches the image features selected to classify the image to the type of picture. For example, a color layout indexing method may be best for outdoor pictures while a region-based indexing approach may be better for indoor pictures. The biomedical image database can be categorized into X-ray, MRI, pathology, graphs, micro-arrays and other features specific to the types of images in the collection.

For general-purpose image libraries and the Web, Wang has classified images into textured vs. non-textured, graph vs. photograph. His approach represents the first time that categories, such as textured vs. non-textured, have been used as a distinguishing feature in image retrieval. In addition, besides using new image features as classification tools, SIMPLIcity uses a similarity measure based on information about the entire image rather than representative segments. In traditional approaches, computer programs may segment one image of a dog, for example, into two regions: the dog and the background. The same program may segment another image of a dog into six regions: the dog's body, the dog's front legs, the dog's rear legs, the dog's eyes, the background and the sky. The inconsistent segmentation makes it harder to make a match.

In SIMPLIcity, an overall "soft similarity" approach reduces the influence of inaccurate segmentation. The most similar region pairs are matched first and then the matching process is "softened" by allowing one region of an image to be matched to several regions of another image. In this way all of the regions of the images are taken into consideration.

"SIMPLIcity is robust to intensity variation, sharpness variations, color distortions, other distortions, cropping, scaling, shifting and rotation,"says Wang. "The system is also easier to use than other region-based retrieval systems."
The work was supported primarily by a research grant from the National Science Foundation's Digital Libraries Initiative and a research fund from the Stanford University Libraries. Additional support came from IBM Almaden Research Center, NEC Research lab, SRI International, Stanford Computer Science Department, Stanford Mathematics Department, Stanford Biomedical Informatics, The Pennsylvania State University and PNC Foundation. The work is on going at Penn State. EDITORS: Dr. Wang is at 814-865-7889 or

Penn State

Related Dog Articles from Brightsurf:

Dog brains do not prefer faces
Even though dogs gaze into man's eyes, dog brains may not process faces as human brains do.

Dog and human brains process faces differently
Researchers discovered striking similarities and differences in how dog and human brains process visual information about others.

Making dog food more delectable by analyzing aromas
Dogs aren't known for being picky about their food, eating the same kibble day after day with relish.

Rates of dog bites in children up during COVID-19 pandemic
Greater rates of Colorado's children are going to the pediatric emergency department as a result of dog bites during the COVID-19 pandemic.

Speech processing hierarchy in the dog brain
Dog brains, just as human brains, process speech hierarchically: intonations at lower, word meanings at higher stages, according to a new study by Hungarian researchers.

When should you neuter your dog to avoid health risks?
A new, 10-year study by researchers at the University of California, Davis, examined 35 dog breeds and found vulnerability from neutering, and the age at which they are neutered, varies greatly depending on the breed.

A scientific measure of dog years
How old is your tail-wagging bundle of joy in human years?

Grassroots dog vaccinations can help stop rabies, but not alone
While scientists are trying to find a vaccine for COVID-19, the rabies virus continues to kill 59,000 people every year.

Digitize your dog into a computer game
Researchers from CAMERA at the University of Bath have developed motion capture technology that enables you to digitise your dog without a motion capture suit and using only one camera.

Yes, your dog wants to rescue you
Imagine you're a dog. Your owner is trapped in a box and is crying out for help.

Read More: Dog News and Dog Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to