Nav: Home

Compact depth sensor inspired by spiders

October 28, 2019

For all our technological advances, nothing beats evolution when it comes to research and development. Take jumping spiders. These small arachnids have impressive depth perception despite their tiny brains, allowing them to accurately pounce on unsuspecting targets from several body lengths away.

Inspired by these spiders, researchers at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have developed a compact and efficient depth sensor that could be used on board microrobots, in small wearable devices, or in lightweight virtual and augmented reality headsets. The device combines a multifunctional, flat metalens with an ultra-efficient algorithm to measure depth in a single shot.

"Evolution has produced a wide variety of optical configurations and vision systems that are tailored to different purposes," said Zhujun Shi, a Ph.D. candidate in the Department of Physics and co-first author of the paper. "Optical design and nanotechnology are finally allowing us to explore artificial depth sensors and other vision systems that are similarly diverse and effective."

The research is published in Proceedings of the National Academy of Sciences (PNAS).

Many of today's depth sensors, such as those in phones, cars and video game consoles, use integrated light sources and multiple cameras to measure distance. Face ID on a smartphone, for example, uses thousands of laser dots to map the contours of the face. This works for large devices with room for batteries and fast computers, but what about small devices with limited power and computation, like smart watches or microrobots?

Evolution, as it turns out, provides a lot of options.

Humans measure depth using stereo vision, meaning when we look at an object, each of our two eyes is collecting a slightly different image. Try this: hold a finger directly in front of your face and alternate opening and closing each of your eyes. See how your finger moves? Our brains take those two images, examine them pixel by pixel and, based on how the pixels shift, calculates the distance to the finger.

"That matching calculation, where you take two images and perform a search for the parts that correspond, is computationally burdensome," said Todd Zickler, the William and Ami Kuan Danoff Professor of Electrical Engineering and Computer Science at SEAS and co-senior author of the study. "Humans have a nice, big brain for those computations but spiders don't."

Jumping spiders have evolved a more efficient system to measure depth. Each principal eye has a few semi-transparent retinae arranged in layers, and these retinae measure multiple images with different amounts of blur. For example, if a jumping spider looks at a fruit fly with one of its principal eyes, the fly will appear sharper in one retina's image and blurrier in another. This change in blur encodes information about the distance to the fly.

In computer vision, this type of distance calculation is known as depth from defocus. But so far, replicating Nature has required large cameras with motorized internal components that can capture differently-focused images over time. This limits the speed and practical applications of the sensor.

That's where the metalens comes in.

Federico Capasso, the Robert L. Wallace Professor of Applied Physics and Vinton Hayes Senior Research Fellow in Electrical Engineering at SEAS and co-senior author of the paper, and his lab have already demonstrated metalenses that can simultaneously produce several images containing different information. Building off that research, the team designed a metalens that can simultaneously produce two images with different blur.

"Instead of using layered retina to capture multiple simultaneous images, as jumping spiders do, the metalens splits the light and forms two differently-defocused images side-by-side on a photosensor," said Shi, who is part of Capasso's lab.

An ultra-efficient algorithm, developed by Zickler's group, then interprets the two images and builds a depth map to represent object distance.

"Being able to design metasurfaces and computational algorithms together is very exciting," said Qi Guo, a Ph.D. candidate in Zickler's lab and co-first author of the paper. "This is new way of creating computational sensors, and it opens the door to many possibilities."

"Metalenses are a game changing technology because of their ability to implement existing and new optical functions much more effciently, faster and with much less bulk and complexity than existing lenses," said Capasso. "Fusing breakthroughs in optical design and computational imaging has led us to this new depth camera that will open up a broad range of opportunities in science and technology."
-end-
This paper was co-authored by Yao-Wei Huang, Emma Alexander, and Cheng-Wei Qiu, of the National University of Singapore. It was supported by Air Force Office of Scientific Research and US National Science Foundation.

Harvard John A. Paulson School of Engineering and Applied Sciences

Related Engineering Articles:

Engineering the meniscus
Damage to the meniscus is common, but there remains an unmet need for improved restorative therapies that can overcome poor healing in the avascular regions.
Artificially engineering the intestine
Short bowel syndrome is a debilitating condition with few treatment options, and these treatments have limited efficacy.
Reverse engineering the fireworks of life
An interdisciplinary team of Princeton researchers has successfully reverse engineered the components and sequence of events that lead to microtubule branching.
New method for engineering metabolic pathways
Two approaches provide a faster way to create enzymes and analyze their reactions, leading to the design of more complex molecules.
Engineering for high-speed devices
A research team from the University of Delaware has developed cutting-edge technology for photonics devices that could enable faster communications between phones and computers.
Breakthrough in blood vessel engineering
Growing functional blood vessel networks is no easy task. Previously, other groups have made networks that span millimeters in size.
Next-gen batteries possible with new engineering approach
Dramatically longer-lasting, faster-charging and safer lithium metal batteries may be possible, according to Penn State research, recently published in Nature Energy.
What can snakes teach us about engineering friction?
If you want to know how to make a sneaker with better traction, just ask a snake.
Engineering a plastic-eating enzyme
Scientists have engineered an enzyme which can digest some of our most commonly polluting plastics, providing a potential solution to one of the world's biggest environmental problems.
A new way to do metabolic engineering
University of Illinois researchers have created a novel metabolic engineering method that combines transcriptional activation, transcriptional interference, and gene deletion, and executes them simultaneously, making the process faster and easier.
More Engineering News and Engineering Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Teaching For Better Humans 2.0
More than test scores or good grades–what do kids need for the future? This hour, TED speakers explore how to help children grow into better humans, both during and after this time of crisis. Guests include educators Richard Culatta and Liz Kleinrock, psychologist Thomas Curran, and writer Jacqueline Woodson.
Now Playing: Science for the People

#556 The Power of Friendship
It's 2020 and times are tough. Maybe some of us are learning about social distancing the hard way. Maybe we just are all a little anxious. No matter what, we could probably use a friend. But what is a friend, exactly? And why do we need them so much? This week host Bethany Brookshire speaks with Lydia Denworth, author of the new book "Friendship: The Evolution, Biology, and Extraordinary Power of Life's Fundamental Bond". This episode is hosted by Bethany Brookshire, science writer from Science News.
Now Playing: Radiolab

Space
One of the most consistent questions we get at the show is from parents who want to know which episodes are kid-friendly and which aren't. So today, we're releasing a separate feed, Radiolab for Kids. To kick it off, we're rerunning an all-time favorite episode: Space. In the 60's, space exploration was an American obsession. This hour, we chart the path from romance to increasing cynicism. We begin with Ann Druyan, widow of Carl Sagan, with a story about the Voyager expedition, true love, and a golden record that travels through space. And astrophysicist Neil de Grasse Tyson explains the Coepernican Principle, and just how insignificant we are. Support Radiolab today at Radiolab.org/donate.