Nav: Home

Camera can watch moving objects around corners

July 29, 2019

David Lindell, a graduate student in electrical engineering at Stanford University, donned a high visibility tracksuit and got to work, stretching, pacing and hopping across an empty room. Through a camera aimed away from Lindell - at what appeared to be a blank wall - his colleagues could watch his every move.

That's because, hidden to the naked eye, he was being scanned by a high powered laser and the single particles of light he reflected onto the walls around him were captured and reconstructed by the camera's advanced sensors and processing algorithm.

"People talk about building a camera that can see as well as humans for applications such as autonomous cars and robots, but we want to build systems that go well beyond that," said Gordon Wetzstein, an assistant professor of electrical engineering at Stanford. "We want to see things in 3D, around corners and beyond the visible light spectrum."

The camera system Lindell tested, which the researchers are presenting at the SIGGRAPH 2019 conference Aug. 1, builds upon previous around-the-corner cameras this team developed. It's able to capture more light from a greater variety of surfaces, see wider and farther away and is fast enough to monitor out-of-sight movement - such as Lindell's calisthenics - for the first time. Someday, the researchers hope superhuman vision systems could help autonomous cars and robots operate even more safely than they would with human guidance.

Practicality and seismology

Keeping their system practical is a high priority for these researchers. The hardware they chose, the scanning and image processing speeds, and the style of imaging are already common in autonomous car vision systems. Previous systems for viewing scenes outside a camera's line of sight relied on objects that either reflect light evenly or strongly. But real-world objects, including shiny cars, fall outside these categories, so this system can handle light bouncing off a range of surfaces, including disco balls, books and intricately textured statues.

Central to their advance was a laser 10,000 times more powerful than what they were using a year ago. The laser scans a wall opposite the scene of interest and that light bounces off the wall, hits the objects in the scene, bounces back to the wall and to the camera sensors. By the time the laser light reaches the camera only specks remain, but the sensor captures every one, sending it along to a highly efficient algorithm, also developed by this team, that untangles these echoes of light to decipher the hidden tableau.

"When you're watching the laser scanning it out, you don't see anything," described Lindell. "With this hardware, we can basically slow down time and reveal these tracks of light. It almost looks like magic."

The system can scan at four frames per second. It can reconstruct a scene at speeds of 60 frames per second on a computer with a graphics processing unit, which enhances graphics processing capabilities.

To advance their algorithm, the team looked to other fields for inspiration. The researchers were particularly drawn to seismic imaging systems - which bounce sound waves off underground layers of Earth to learn what's beneath the surface - and reconfigured their algorithm to likewise interpret bouncing light as waves emanating from the hidden objects. The result was the same high-speed and low memory usage with improvements in their abilities to see large scenes containing various materials.

"There are many ideas being used in other spaces - seismology, imaging with satellites, synthetic aperture radar - that are applicable to looking around corners," said Matthew O'Toole, an assistant professor at Carnegie Mellon University who was previously a postdoctoral fellow in Wetzstein's lab. "We're trying to take a little bit from these fields and we'll hopefully be able to give something back to them at some point."

Humble steps

Being able to see real-time movement from otherwise invisible light bounced around a corner was a thrilling moment for this team but a practical system for autonomous cars or robots will require further enhancements.

"It's very humble steps. The movement still looks low-resolution and it's not super-fast but compared to the state-of-the-art last year it is a significant improvement," said Wetzstein. "We were blown away the first time we saw these results because we've captured data that nobody's seen before."

The team hopes to move toward testing their system on autonomous research cars, while looking into other possible applications, such as medical imaging that can see through tissues. Among other improvements to speed and resolution, they'll also work at making their system even more versatile to address challenging visual conditions that drivers encounter, such as fog, rain, sandstorms and snow.
-end-
To read all stories about Stanford science, subscribe to the biweekly Stanford Science Digest.

Wetzstein is also a member of Stanford Bio-X and the Wu Tsai Neurosciences Institute.

This work was funded by a Stanford Graduate Fellowship, the National Science Foundation, a Terman Faculty Fellowship, a Sloan Fellowship, the King Abdullah University of Science and Technology, the Center for Automotive Research at Stanford, the Defense Advanced Research Projects Agency, the U.S. Army Research Laboratory.

Stanford University

Related Science Articles:

75 science societies urge the education department to base Title IX sexual harassment regulations on evidence and science
The American Educational Research Association (AERA) and the American Association for the Advancement of Science (AAAS) today led 75 scientific societies in submitting comments on the US Department of Education's proposed changes to Title IX regulations.
Science/Science Careers' survey ranks top biotech, biopharma, and pharma employers
The Science and Science Careers' 2018 annual Top Employers Survey polled employees in the biotechnology, biopharmaceutical, pharmaceutical, and related industries to determine the 20 best employers in these industries as well as their driving characteristics.
Science in the palm of your hand: How citizen science transforms passive learners
Citizen science projects can engage even children who previously were not interested in science.
Applied science may yield more translational research publications than basic science
While translational research can happen at any stage of the research process, a recent investigation of behavioral and social science research awards granted by the NIH between 2008 and 2014 revealed that applied science yielded a higher volume of translational research publications than basic science, according to a study published May 9, 2018 in the open-access journal PLOS ONE by Xueying Han from the Science and Technology Policy Institute, USA, and colleagues.
Prominent academics, including Salk's Thomas Albright, call for more science in forensic science
Six scientists who recently served on the National Commission on Forensic Science are calling on the scientific community at large to advocate for increased research and financial support of forensic science as well as the introduction of empirical testing requirements to ensure the validity of outcomes.
World Science Forum 2017 Jordan issues Science for Peace Declaration
On behalf of the coordinating organizations responsible for delivering the World Science Forum Jordan, the concluding Science for Peace Declaration issued at the Dead Sea represents a global call for action to science and society to build a future that promises greater equality, security and opportunity for all, and in which science plays an increasingly prominent role as an enabler of fair and sustainable development.
PETA science group promotes animal-free science at society of toxicology conference
The PETA International Science Consortium Ltd. is presenting two posters on animal-free methods for testing inhalation toxicity at the 56th annual Society of Toxicology (SOT) meeting March 12 to 16, 2017, in Baltimore, Maryland.
Citizen Science in the Digital Age: Rhetoric, Science and Public Engagement
James Wynn's timely investigation highlights scientific studies grounded in publicly gathered data and probes the rhetoric these studies employ.
Science/Science Careers' survey ranks top biotech, pharma, and biopharma employers
The Science and Science Careers' 2016 annual Top Employers Survey polled employees in the biotechnology, biopharmaceutical, pharmaceutical, and related industries to determine the 20 best employers in these industries as well as their driving characteristics.
Three natural science professors win TJ Park Science Fellowship
Professor Jung-Min Kee (Department of Chemistry, UNIST), Professor Kyudong Choi (Department of Mathematical Sciences, UNIST), and Professor Kwanpyo Kim (Department of Physics, UNIST) are the recipients of the Cheong-Am (TJ Park) Science Fellowship of the year 2016.
More Science News and Science Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Power Of Spaces
How do spaces shape the human experience? In what ways do our rooms, homes, and buildings give us meaning and purpose? This hour, TED speakers explore the power of the spaces we make and inhabit. Guests include architect Michael Murphy, musician David Byrne, artist Es Devlin, and architect Siamak Hariri.
Now Playing: Science for the People

#576 Science Communication in Creative Places
When you think of science communication, you might think of TED talks or museum talks or video talks, or... people giving lectures. It's a lot of people talking. But there's more to sci comm than that. This week host Bethany Brookshire talks to three people who have looked at science communication in places you might not expect it. We'll speak with Mauna Dasari, a graduate student at Notre Dame, about making mammals into a March Madness match. We'll talk with Sarah Garner, director of the Pathologists Assistant Program at Tulane University School of Medicine, who takes pathology instruction out of...
Now Playing: Radiolab

What If?
There's plenty of speculation about what Donald Trump might do in the wake of the election. Would he dispute the results if he loses? Would he simply refuse to leave office, or even try to use the military to maintain control? Last summer, Rosa Brooks got together a team of experts and political operatives from both sides of the aisle to ask a slightly different question. Rather than arguing about whether he'd do those things, they dug into what exactly would happen if he did. Part war game part choose your own adventure, Rosa's Transition Integrity Project doesn't give us any predictions, and it isn't a referendum on Trump. Instead, it's a deeply illuminating stress test on our laws, our institutions, and on the commitment to democracy written into the constitution. This episode was reported by Bethel Habte, with help from Tracie Hunte, and produced by Bethel Habte. Jeremy Bloom provided original music. Support Radiolab by becoming a member today at Radiolab.org/donate.     You can read The Transition Integrity Project's report here.