Nav: Home

Deep3DFly: the deep-learning way to design fly-like robots

October 09, 2019

"Just think about what a fly can do," says Professor Pavan Ramdya, whose lab at EPFL's Brain Mind Institute, with the lab of Professor Pascal Fua in EPFL's Institute for Computer Science, led the study. "A fly can climb across terrain that a wheeled robot would not be able to."

Flies aren't exactly endearing to humans. We rightly associate them with less-than-appetizing experiences in our daily lives. But there is an unexpected path to redemption: Robots. It turns out that flies have some features and abilities that can inform a new design for robotic systems.

"Unlike most vertebrates, flies can climb nearly any terrain," says Ramdya. "They can stick to walls and ceilings because they have adhesive pads and claws on the tips of their legs. This allows them to basically go anywhere. That's interesting also because if you can rest on any surface, you can manage your energy expenditure by waiting for the right moment to act."

It was this vision of extracting the principles that govern fly behavior to inform the design of robots that drove the development of DeepFly3D, a motion-capture system for the fly Drosophila melanogaster, a model organism that is nearly ubiquitously used across biology.

In Ramdya's experimental setup, a fly walks on top of a tiny floating ball - like a miniature treadmill - while seven cameras record its every movement. The fly's top side is glued onto an unmovable stage so that it always stays in place while walking on the ball. Nevertheless, the fly "believes" that it is moving freely.

The collected camera images are then processed by DeepFly3D, a deep-learning software developed by Semih Günel, a PhD student working with both Ramdya's and Fua's labs. "This is a fine example of where an interdisciplinary collaboration was necessary and transformative," says Ramdya. "By leveraging computer science and neuroscience, we've tackled a long-standing challenge."

What's special about DeepFly3D is that is can infer the 3D pose of the fly - or even other animals - meaning that it can automatically predict and make behavioral measurements at unprecedented resolution for a variety of biological applications. The software doesn't need to be calibrated manually and it uses camera images to automatically detect and correct any errors it makes in its calculations of the fly's pose. Finally, it also uses active learning to improve its own performance.

DeepFly3D opens up a way to efficiently and accurately model the movements, poses, and joint angles of a fruit fly in three dimensions. This may inspire a standard way to automatically model 3D pose in other organisms as well.

"The fly, as a model organism, balances tractability and complexity very well," says Ramdya. "If we learn how it does what it does, we can have important impact on robotics and medicine and, perhaps most importantly, we can gain these insights in a relatively short period of time."
-end-
Funding

Microsoft (JRC project)
Swiss National Science Foundation (Eccellenza grant)
EPFL (iPhD grant)
Swiss Government Excellence Postdoctoral Scholarship

Reference

Semih Günel, Helge Rhodin, Daniel Morales, João H. Campagnolo, Pavan Ramdya, Pascal Fua. DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila. eLife 04 October 2019;8:e48571. DOI: 10.7554/eLife.48571

Ecole Polytechnique Fédérale de Lausanne

Related Robots Articles:

Robots popular with older adults
A new study by psychologists from the University of Jena (Germany) does not confirm that robot skepticism among elder people is often suspected in science.
Showing robots how to do your chores
By observing humans, robots learn to perform complex tasks, such as setting a table.
Designing better nursing care with robots
Robots are becoming an increasingly important part of human care, according to researchers based in Japan.
Darn you, R2! When can we blame robots?
A recent study finds that people are likely to blame robots for workplace accidents, but only if they believe the robots are autonomous.
Robots need a new philosophy to get a grip
Robots need to know the reason why they are doing a job if they are to effectively and safely work alongside people in the near future.
How can robots land like birds?
Birds can perch on a wide variety of surfaces, thick or thin, rough or slick.
Soft robots for all
Each year, soft robots gain new abilities. They can jump, squirm, and grip.
The robots that dementia caregivers want: robots for joy, robots for sorrow
A team of scientists spent six months co-designing robots with informal caregivers for people with dementia, such as family members.
Faster robots demoralize co-workers
A Cornell University-led team has found that when robots are beating humans in contests for cash prizes, people consider themselves less competent and expend slightly less effort -- and they tend to dislike the robots.
Increasing skepticism against robots
In Europe, people are more reserved regarding robots than they were five years ago.
More Robots News and Robots Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Reinvention
Change is hard, but it's also an opportunity to discover and reimagine what you thought you knew. From our economy, to music, to even ourselves–this hour TED speakers explore the power of reinvention. Guests include OK Go lead singer Damian Kulash Jr., former college gymnastics coach Valorie Kondos Field, Stockton Mayor Michael Tubbs, and entrepreneur Nick Hanauer.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Dispatch 6: Strange Times
Covid has disrupted the most basic routines of our days and nights. But in the middle of a conversation about how to fight the virus, we find a place impervious to the stalled plans and frenetic demands of the outside world. It's a very different kind of front line, where urgent work means moving slow, and time is marked out in tiny pre-planned steps. Then, on a walk through the woods, we consider how the tempo of our lives affects our minds and discover how the beats of biology shape our bodies. This episode was produced with help from Molly Webster and Tracie Hunte. Support Radiolab today at Radiolab.org/donate.