Nav: Home

How to train your robot (to feed you dinner)

March 11, 2019

About 1 million adults in the United States need someone to help them eat, according to census data from 2010.

It's a time-consuming and often awkward task, one largely done out of necessity rather than choice.

Researchers at the University of Washington are working on a robotic system that can help make it easier. After identifying different foods on a plate, the robot can strategize how to use a fork to pick up and deliver the desired bite to a person's mouth.

The team published its results in a series of papers: One was recently published in IEEE Robotics and Automation Letters, while the other will be presented March 13 at the ACM/IEEE International Conference on Human-Robot Interaction in South Korea.

"Being dependent on a caregiver to feed every bite every day takes away a person's sense of independence," said corresponding author Siddhartha Srinivasa, the Boeing Endowed Professor in the UW's Paul G. Allen School of Computer Science & Engineering. "Our goal with this project is to give people a bit more control over their lives."

The idea was to develop an autonomous feeding system that would be attached to people's wheelchairs and feed people whatever they wanted to eat.

"When we started the project we realized: There are so many ways that people can eat a piece of food depending on its size, shape or consistency. How do we start?" said co-author Tapomayukh Bhattacharjee, a postdoctoral research associate in the Allen School. "So we set up an experiment to see how humans eat common foods like grapes and carrots."

The researchers arranged plates with about a dozen different kinds of food, ranging in consistency from hard carrots to soft bananas. The plates also included foods like tomatoes and grapes, which have a tough skin and soft insides. Then the team gave volunteers a fork and asked them to pick up different pieces of food and feed them to a mannequin. The fork contained a sensor to measure how much force people used when they picked up food.

The volunteers used various strategies to pick up food with different consistencies. For example, people skewered soft items like bananas at an angle to keep them from slipping off the fork. For items like carrots and grapes, the volunteers tended to use wiggling motions to increase the force and spear each bite.

"People seemed to use different strategies not just based on the size and shape of the food but also how hard or soft it is. But do we actually need to do that?" Bhattacharjee said. "We decided to do an experiment with the robot where we had it skewer food until the fork reached a certain depth inside, regardless of the type of food."

The robot used the same force-and-skewering strategy to try to pick up all the pieces of food, regardless of their consistency. It was able to pick up hard foods, but it struggled with soft foods and those with tough skins and soft insides. So robots, like humans, need to adjust how much force and angle they use to pick up different kinds of food.

The team also noted that the acts of picking up a piece of food and feeding it to someone are not independent of each other. Volunteers often would specifically orient a piece of food on the fork so that it could be eaten easily.

"You can pick up a carrot stick by skewering it in the center of the stick, but it will be difficult for a person to eat," Bhattacharjee said. "On the other hand, if you pick it up on one of the ends and then tilt the carrot toward someone's mouth, it's easier to take a bite."

To design a skewering and feeding strategy that changes based on the food item, the researchers combined two different algorithms. First they used an object-detection algorithm called RetinaNet, which scans the plate, identifies the types of food on it and places a frame around each item.

Then they developed SPNet, an algorithm that examines the type of food in a specific frame and tells the robot the best way to pick up the food. For example, SPNet tells the robot to skewer a strawberry or a slice of banana in the middle, and spear carrots at one of the two ends.

The team had the robot pick up pieces of food and feed them to volunteers using SPNet or a more uniform strategy: an approach that skewered the center of each food item regardless of what it was. SPNet's varying strategies outperformed or performed the same as the uniform approach for all the food.

"Many engineering challenges are not picky about their solutions, but this research is very intimately connected with people," Srinivasa said. "If we don't take into account how easy it is for a person to take a bite, then people might not be able to use our system. There's a universe of types of food out there, so our biggest challenge is to develop strategies that can deal with all of them."

The team is currently working with the Taskar Center for Accessible Technology to get feedback from caregivers and patients in assisted living facilities on how to improve the system to match people's needs.

"Ultimately our goal is for our robot to help people have their lunch or dinner on their own," Srinivasa said. "But the point is not to replace caregivers: We want to empower them. With a robot to help, the caregiver can set up the plate, and then do something else while the person eats."
-end-
Co-authors for the first paper include doctoral student Gilwoo Lee and research scientist Hanjun Song, both at the Allen School. Co-authors on the second paper include Daniel Gallenberger, a master's student at Technische Universität München in Germany who completed this research while at the UW, and Youngsun Kim, a research scientist at the Allen School. This work was first debuted at the recent NeurIPS conference, where it won a best demo award. This research was funded by the National Institutes of Health, the National Science Foundation, the Office of Naval Research, the Robotics Collaborative Technology Alliance, Amazon and Honda.

For more information, contact Srinivasa at siddh@cs.washington.edu.

Grant numbers: R01EB019335, 1544797, 1637748

University of Washington

Related Robot Articles:

A robot with a firm yet gentle grasp
Human hands are skilled at manipulating a range of objects.
You can't squash this roach-inspired robot
A new insect-sized robot created by researchers at the University of California, Berkeley, can scurry across the floor at nearly the speed of a darting cockroach -- and it's nearly as hardy as a cockroach, too: Try to squash this robot under your foot, and more than likely, it will just keep going.
In the shoes of a robot: The future approaches
Identifying with someone is an exercise that makes us understand them deeply, empathize with them, and helps us overcome mistrust and prejudice.
Now your phone can become a robot that does the boring work
Purdue University researchers have developed a smartphone app that allows a user to easily program any robot to perform a task, dramatically bringing down the costs of building and programming mobile robots.
Better together: human and robot co-workers
More and more processes are being automated and digitised. Self-driving delivery vehicles, such as forklifts, are finding their way into many areas -- and companies are reporting potential time and cost savings.
Robot therapists need rules
Interactions with artificial intelligence (AI) will become an increasingly common aspect of our lives.
Hummingbird robot uses AI to soon go where drones can't
Purdue University researchers have engineered flying robots that behave like hummingbirds, trained by machine learning algorithms based on various techniques the bird uses naturally every day.
How to train your robot (to feed you dinner)
Researchers at the University of Washington have developed a robotic system that can feed people who need someone to help them eat.
The first walking robot that moves without GPS
Desert ants are extraordinary solitary navigators. Researchers at CNRS and Aix-Marseille University, in the Institut des Sciences du Mouvement -- Étienne Jules Marey (ISM), were inspired by these ants as they designed AntBot, the first walking robot that can explore its environment randomly and go home automatically, without GPS or mapping.
Getting a grip on human-robot cooperation
The study, published in Science Robotics, is a collaboration between The BioRobotics Institute (Scuola Superiore Sant'Anna, Pisa, Italy) and the Australian Centre for Robotic Vision (Queensland University of Technology, Brisbane, Australia).
More Robot News and Robot Current Events

Top Science Podcasts

We have hand picked the top science podcasts of 2019.
Now Playing: TED Radio Hour

Risk
Why do we revere risk-takers, even when their actions terrify us? Why are some better at taking risks than others? This hour, TED speakers explore the alluring, dangerous, and calculated sides of risk. Guests include professional rock climber Alex Honnold, economist Mariana Mazzucato, psychology researcher Kashfia Rahman, structural engineer and bridge designer Ian Firth, and risk intelligence expert Dylan Evans.
Now Playing: Science for the People

#541 Wayfinding
These days when we want to know where we are or how to get where we want to go, most of us will pull out a smart phone with a built-in GPS and map app. Some of us old timers might still use an old school paper map from time to time. But we didn't always used to lean so heavily on maps and technology, and in some remote places of the world some people still navigate and wayfind their way without the aid of these tools... and in some cases do better without them. This week, host Rachelle Saunders...
Now Playing: Radiolab

Dolly Parton's America: Neon Moss
Today on Radiolab, we're bringing you the fourth episode of Jad's special series, Dolly Parton's America. In this episode, Jad goes back up the mountain to visit Dolly's actual Tennessee mountain home, where she tells stories about her first trips out of the holler. Back on the mountaintop, standing under the rain by the Little Pigeon River, the trip triggers memories of Jad's first visit to his father's childhood home, and opens the gateway to dizzying stories of music and migration. Support Radiolab today at Radiolab.org/donate.