Would You Trust A Robot To Sort Chocolates?

March 26, 1997

A packaging robot, working in a candy factory in the near future, aims its electronic eye at a tray crammed with chocolates. The robot must fill a box with a specific mix of candy, but how can it tell a cream-filled morsel from a chocolate-covered peanut cluster?

A new object recognition technique developed at The Johns Hopkins University might help this robot hold onto its job. Using three lights, a video camera and a computer, researcher Elli Angelopoulou has devised a new way to transform visual images into electronic "signatures." Each is a distinctive series of 11 numbers. By comparing these signatures, a computer can tell how closely two objects resemble one another. For instance, a smooth, dome-shaped candy and a bumpy nut cluster would have far different signatures.

Angelopoulou, a doctoral student in the Department of Computer Science and an admitted chocoholic, has used the technique to identify coffee mugs, disposable razors, children's toys--and, yes, chocolate candies. It can even distinguish between two objects that have identical shapes but different colors. Her technique is primarily for recognizing objects with curved surfaces, a task that has long troubled computer vision scientists. The next step, Hopkins researchers say, is for robotics experts to incorporate Angelopoulou's breakthrough into a mechanical packaging machine. "As part of an industrial object recognition system, this technique could expand a robot's capability," says Larry Wolff, an associate professor of computer science who supervises the Computer Vision Laboratory in Hopkins' Whiting School of Engineering. Wolff and James P. Williams, a doctoral student who also is a member of the Computer Vision Laboratory group, contributed to the development of this new object recognition system.

A mechanical device that interacts with its environment must have some sort of visual information about its surroundings, says Angelopoulou (pronounced ann-jell-oh-POOH'-looh). "A robot or a vehicle that moves by itself needs to 'see' where it's going," she says. "We need a way for it to recognize objects." Scientists have invented electronic ways to identify some objects the way people often do: by following the edges until a rectangular shape, for example, is recognized as a door. But such systems do not work well on curved objects, such as vases, that have few edges to follow.

In 1995, Angelopoulou decided on a different approach: measuring the light that bounces off an object and into the lens of a video camera. "My goal was to find an object recognition technique by looking at the intensity values," she says. "That meant I was not going to look at the edges. I was going to look at every pixel and the value the camera recorded. I believed I could collect enough information for an object signature just by looking at the intensity values recorded by the camera under three lighting conditions."

Two years later, here's how her system works: Angelopoulou aims one lamp in the same direction as her camera lens and positions the other two on opposite sides of an object such as a coffee mug. She shoots the mug three times, first switching one lamp on and off, then repeating the process with the other two. The camera captures this mug in about 10,000 pixels--the tiny points on a video screen that blend together and form an image. For each pixel, Angelopoulou's software computes a DeCov value, short for determinant of the photometric covariance matrix. These values provide critical information about how the object curves. Within a few minutes, the computer sorts and reduces these values to a distinctive series of 11 numbers. These become the signature for the mug. If another picture produces the same or nearly the same series of numbers, the computer "recognizes" that object, too, as a coffee mug. Different colors reflect light differently, so the system can also separate blue and red mugs that have the same shape.

This system is less sensitive to the electronic "noise" or interference that hampers other computer vision systems, Angelopoulou says. And, unlike those systems, it does not require extremely precise placement and measurement of lighting. But the process does have limitations. It will recognize a mug from many upright angles, but the computer will be fooled if the camera looks at the flat bottom of the cup. Also, the computer will recognize identical objects only if the lighting remains unchanged.

A factory could provide such controlled lighting, and Angelopoulou hopes other scientists will someday install her system in a robot that could sort and pack mechanical parts, toys or even chocolates. "Usually, what we do in smaller labs is just try to solve a small part of the problem," she says. "Then we hope someone else will integrate it into a device that solves a bigger part of the problem."

Note:A black-and-white or color image is available to reporters on request. Contact Phil Sneiderman at the phone number or e-mail address above.

Johns Hopkins University

Related Robot Articles from Brightsurf:

Autonomous robot plays with NanoLEGO
Atoms and molecules behave in a completely different way to macroscopic objects and each brick requires its own ''instruction manual''.

Robot technology for everyone or only for the average person?
Robot technology is being used more and more in health rehabilitation and in working life.

Soft robot, unplugged
It's balloon art on steroids: a pneumatic, shape-changing soft robot capable of navigating its environment without requiring a tether to a stationary power source.

Trash talk hurts, even when it comes from a robot
Trash talking has a long and colorful history of flustering game opponents, and now researchers at Carnegie Mellon University have demonstrated that discouraging words can be perturbing even when uttered by a robot.

A robot with a firm yet gentle grasp
Human hands are skilled at manipulating a range of objects.

You can't squash this roach-inspired robot
A new insect-sized robot created by researchers at the University of California, Berkeley, can scurry across the floor at nearly the speed of a darting cockroach -- and it's nearly as hardy as a cockroach, too: Try to squash this robot under your foot, and more than likely, it will just keep going.

In the shoes of a robot: The future approaches
Identifying with someone is an exercise that makes us understand them deeply, empathize with them, and helps us overcome mistrust and prejudice.

Now your phone can become a robot that does the boring work
Purdue University researchers have developed a smartphone app that allows a user to easily program any robot to perform a task, dramatically bringing down the costs of building and programming mobile robots.

Better together: human and robot co-workers
More and more processes are being automated and digitised. Self-driving delivery vehicles, such as forklifts, are finding their way into many areas -- and companies are reporting potential time and cost savings.

Robot therapists need rules
Interactions with artificial intelligence (AI) will become an increasingly common aspect of our lives.

Read More: Robot News and Robot Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.