Nav: Home

Seeing through a robot's eyes helps those with profound motor impairments

March 15, 2019

An interface system that uses augmented reality technology could help individuals with profound motor impairments operate a humanoid robot to feed themselves and perform routine personal care tasks such as scratching an itch and applying skin lotion. The web-based interface displays a "robot's eye view" of surroundings to help users interact with the world through the machine.

The system, described March 15 in the journal PLOS ONE, could help make sophisticated robots more useful to people who do not have experience operating complex robotic systems. Study participants interacted with the robot interface using standard assistive computer access technologies -- such as eye trackers and head trackers -- that they were already using to control their personal computers.

The paper reported on two studies showing how such "robotic body surrogates" - which can perform tasks similar to those of humans - could improve the quality of life for users. The work could provide a foundation for developing faster and more capable assistive robots.

"Our results suggest that people with profound motor deficits can improve their quality of life using robotic body surrogates," said Phillip Grice, a recent Georgia Institute of Technology Ph.D. graduate who is first author of the paper. "We have taken the first step toward making it possible for someone to purchase an appropriate type of robot, have it in their home and derive real benefit from it."

Grice and Professor Charlie Kemp from the Wallace H. Coulter Department of Biomedical Engineering at Georgia Tech and Emory University used a PR2 mobile manipulator manufactured by Willow Garage for the two studies. The wheeled robot has 20 degrees of freedom, with two arms and a "head," giving it the ability to manipulate objects such as water bottles, washcloths, hairbrushes and even an electric shaver.

"Our goal is to give people with limited use of their own bodies access to robotic bodies so they can interact with the world in new ways," said Kemp.

In their first study, Grice and Kemp made the PR2 available across the internet to a group of 15 participants with severe motor impairments. The participants learned to control the robot remotely, using their own assistive equipment to operate a mouse cursor to perform a personal care task. Eighty percent of the participants were able to manipulate the robot to pick up a water bottle and bring it to the mouth of a mannequin.

"Compared to able-bodied persons, the capabilities of the robot are limited," Grice said. "But the participants were able to perform tasks effectively and showed improvement on a clinical evaluation that measured their ability to manipulate objects compared to what they would have been able to do without the robot."

In the second study, the researchers provided the PR2 and interface system to Henry Evans, a California man who has been helping Georgia Tech researchers study and improve assistive robotic systems since 2011. Evans, who has very limited control of his body, tested the robot in his home for seven days and not only completed tasks, but also devised novel uses combining the operation of both robot arms at the same time - using one arm to control a washcloth and the other to use a brush.

"The system was very liberating to me, in that it enabled me to independently manipulate my environment for the first time since my stroke," said Evans. "With respect to other people, I was thrilled to see Phil get overwhelmingly positive results when he objectively tested the system with 15 other people."

The researchers were pleased that Evans developed new uses for the robot, combining motion of the two arms in ways they had not expected.

"When we gave Henry free access to the robot for a week, he found new opportunities for using it that we had not anticipated," said Grice. "This is important because a lot of the assistive technology available today is designed for very specific purposes. What Henry has shown is that this system is powerful in providing assistance and empowering users. The opportunities for this are potentially very broad."

The interface allowed Evans to care for himself in bed over an extended period of time. "The most helpful aspect of the interface system was that I could operate the robot completely independently, with only small head movements using an extremely intuitive graphical user interface," Evans said.

The web-based interface shows users what the world looks like from cameras located in the robot's head. Clickable controls overlaid on the view allow the users to move the robot around in a home or other environment and control the robot's hands and arms. When users move the robot's head, for instance, the screen displays the mouse cursor as a pair of eyeballs to show where the robot will look when the user clicks. Clicking on a disc surrounding the robotic hands allows users to select a motion. While driving the robot around a room, lines following the cursor on the interface indicate the direction it will travel.

Building the interface around the actions of a simple single-button mouse allows people with a range of disabilities to use the interface without lengthy training sessions.

"Having an interface that individuals with a wide range of physical impairments can operate means we can provide access to a broad range of people, a form of universal design," Grice noted. "Because of its capability, this is a very complex system, so the challenge we had to overcome was to make it accessible to individuals who have very limited control of their own bodies."

While the results of the study demonstrated what the researchers had set out to do, Kemp agrees that improvements can be made. The existing system is slow, and mistakes made by users can create significant setbacks. Still, he said, "People could use this technology today and really benefit from it."

The cost and size of the PR2 would need to be significantly reduced for the system to be commercially viable, Evans suggested. Kemp says these studies point the way to a new type of assistive technology.

"It seems plausible to me based on this study that robotic body surrogates could provide significant benefits to users," Kemp added.
-end-
This work was supported by the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), grant 90RE5016-01-00 via RERC TechSAge, National Science Foundation Award IIS-1150157, by a National Science Foundation Graduate Research Fellowship Program Award, and the Residential Care Facilities for the Elderly of Fulton County Scholar Award.

Kemp is a cofounder, a board member, an equity holder, and the CTO of Hello Robot Inc., which is developing products related to this research. This research could affect his personal financial status. The terms of this arrangement have been reviewed and approved by Georgia Tech in accordance with its conflict of interest policies.

CITATION: Phillip M. Grice and Charles C. Kemp, "In-home and remote use of robotic body surrogates by people with profound motor deficits" (PLOS ONE 2019).

Georgia Institute of Technology

Related Robot Articles:

Autonomous robot plays with NanoLEGO
Atoms and molecules behave in a completely different way to macroscopic objects and each brick requires its own ''instruction manual''.
Robot technology for everyone or only for the average person?
Robot technology is being used more and more in health rehabilitation and in working life.
Soft robot, unplugged
It's balloon art on steroids: a pneumatic, shape-changing soft robot capable of navigating its environment without requiring a tether to a stationary power source.
Trash talk hurts, even when it comes from a robot
Trash talking has a long and colorful history of flustering game opponents, and now researchers at Carnegie Mellon University have demonstrated that discouraging words can be perturbing even when uttered by a robot.
A robot with a firm yet gentle grasp
Human hands are skilled at manipulating a range of objects.
You can't squash this roach-inspired robot
A new insect-sized robot created by researchers at the University of California, Berkeley, can scurry across the floor at nearly the speed of a darting cockroach -- and it's nearly as hardy as a cockroach, too: Try to squash this robot under your foot, and more than likely, it will just keep going.
In the shoes of a robot: The future approaches
Identifying with someone is an exercise that makes us understand them deeply, empathize with them, and helps us overcome mistrust and prejudice.
Now your phone can become a robot that does the boring work
Purdue University researchers have developed a smartphone app that allows a user to easily program any robot to perform a task, dramatically bringing down the costs of building and programming mobile robots.
Better together: human and robot co-workers
More and more processes are being automated and digitised. Self-driving delivery vehicles, such as forklifts, are finding their way into many areas -- and companies are reporting potential time and cost savings.
Robot therapists need rules
Interactions with artificial intelligence (AI) will become an increasingly common aspect of our lives.
More Robot News and Robot Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Power Of Spaces
How do spaces shape the human experience? In what ways do our rooms, homes, and buildings give us meaning and purpose? This hour, TED speakers explore the power of the spaces we make and inhabit. Guests include architect Michael Murphy, musician David Byrne, artist Es Devlin, and architect Siamak Hariri.
Now Playing: Science for the People

#576 Science Communication in Creative Places
When you think of science communication, you might think of TED talks or museum talks or video talks, or... people giving lectures. It's a lot of people talking. But there's more to sci comm than that. This week host Bethany Brookshire talks to three people who have looked at science communication in places you might not expect it. We'll speak with Mauna Dasari, a graduate student at Notre Dame, about making mammals into a March Madness match. We'll talk with Sarah Garner, director of the Pathologists Assistant Program at Tulane University School of Medicine, who takes pathology instruction out of...
Now Playing: Radiolab

What If?
There's plenty of speculation about what Donald Trump might do in the wake of the election. Would he dispute the results if he loses? Would he simply refuse to leave office, or even try to use the military to maintain control? Last summer, Rosa Brooks got together a team of experts and political operatives from both sides of the aisle to ask a slightly different question. Rather than arguing about whether he'd do those things, they dug into what exactly would happen if he did. Part war game part choose your own adventure, Rosa's Transition Integrity Project doesn't give us any predictions, and it isn't a referendum on Trump. Instead, it's a deeply illuminating stress test on our laws, our institutions, and on the commitment to democracy written into the constitution. This episode was reported by Bethel Habte, with help from Tracie Hunte, and produced by Bethel Habte. Jeremy Bloom provided original music. Support Radiolab by becoming a member today at Radiolab.org/donate.     You can read The Transition Integrity Project's report here.