Nav: Home

New, more realistic simulator will improve self-driving vehicle safety before road testing

March 27, 2019

University of Maryland computer scientist Dinesh Manocha, in collaboration with a team of colleagues from Baidu Research and the University of Hong Kong, has developed a photo-realistic simulation system for training and validating self-driving vehicles. The new system provides a richer, more authentic simulation than current systems that use game engines or high-fidelity computer graphics and mathematically rendered traffic patterns.

Their system, called Augmented Autonomous Driving Simulation (AADS), could make self-driving technology easier to evaluate in the lab while also ensuring more reliable safety before expensive road testing begins.

The scientists described their methodology in a research paper published March 27, 2019 in the journal Science Robotics.

"This work represents a new simulation paradigm in which we can test the reliability and safety of automatic driving technology before we deploy it on real cars and test it on the highways or city roads," said Manocha, one of the paper's corresponding authors, and a professor with joint appointments in computer science, electrical and computer engineering, and the University of Maryland Institute for Advanced Computer Studies.

One potential benefit of self-driving cars is that they could be safer than human drivers who are prone to distraction, fatigue and emotional decisions that lead to mistakes. But to ensure safety, autonomous vehicles must evaluate and respond to the driving environment without fail. Given the innumerable situations that a car might encounter on the road, an autonomous driving system requires hundreds of millions of miles worth of test drives under challenging conditions to demonstrate reliability.

While that could take decades to accomplish on the road, preliminary evaluations could be conducted quickly, efficiently and more safely by computer simulations that accurately represent the real world and model the behavior of surrounding objects. Current state-of-the art simulation systems described in scientific literature fall short in portraying photo-realistic environments and presenting real-world traffic flow patterns or driver behaviors.

AADS is a data-driven system that more accurately represents the inputs a self-driving car would receive on the road. Self-driving cars rely on a perception module, which receives and interprets information about the real world, and a navigation module that makes decisions, such as where to steer or whether to break or accelerate, based on the perception module.

In the real world, the perception module of a self-driving car typically receives input from cameras and lidar sensors, which use pulses of light to measure distances of surrounding. In current simulator technology, the perception module receives input from computer-generated imagery and mathematically modeled movement patterns for pedestrians, bicycles, and other cars. It is a relatively crude representation of the real world. It is also expensive and time- consuming to create because computer-generated imagery models must be hand generated.

The AADS system combines photos, videos, and lidar point clouds--which are like 3D shape renderings--with real-world trajectory data for pedestrians, bicycles, and other cars. These trajectories can be used to predict the driving behavior and future positions of other vehicles or pedestrians on the road for safer navigation.

"We are rendering and simulating the real world visually, using videos and photos," said Manocha, "but also we're capturing real behavior and patterns of movement. The way humans drive is not easy to capture by mathematical models and laws of physics. So, we extracted data about real trajectories from all the video we had available, and we modeled driving behaviors using social science methodologies. This data-driven approach has given us a much more realistic and beneficial traffic simulator."

The scientists had a long-standing challenge to overcome in using real video imagery and lidar data for their simulation: Every scene must respond to a self-driving car's movements, even though those movements may not have been captured by the original camera or lidar sensor. Whatever angle or viewpoint is not captured by a photo or video has to be rendered or simulated using prediction methods. This is why simulation technology has always relied so heavily on computer-generated graphics and physics-based prediction techniques.

To overcome this challenge, the researchers developed technology that isolates the various components of a real-world street scene and renders them as individual elements that can be resynthesized to create a multitude of photo-realistic driving scenarios.

With AADS, vehicles and pedestrians can be lifted from one environment and placed into another with the proper lighting and movement patterns. Roads can be recreated with different levels of traffic. Multiple viewing angles of every scene provide more realistic perspectives during lane changes and turns. In addition, advanced image processing technology enables smooth transitions and reduces distortion compared with other video simulation techniques. The image processing techniques are also used to extract trajectories, and thereby model driver behaviors.

"Because we're using real-world video and real-world movements, our perception module has more accurate information than previous methods," Manocha said. "And then, because of the realism of the simulator, we can better evaluate navigation strategies of an autonomous driving system."

Manocha said that by publishing this work, the scientists hope some of the corporations developing self-driving vehicles might incorporate the same data-driven approach to improve their own simulators for testing and evaluating autonomous driving systems.
-end-
A video demonstration of the AADS system can be seen here: https://www.youtube.com/watch?v=OfxqXhcMH5g&feature=youtu.be

The research paper, "AADS: Augmented autonomous driving simulation using data-driven algorithms," W. Li, C. W. Pan, R. Zhang, J. P. Ren, Y. X. Ma, J. Fang, F. L. Yan, Q. C. Geng, X. Y. Huang, H. J. Gong, W. W. Xu, G. P. Wang, D. Manocha, R. G. Yang, was published in the journal Science Robotics on March 27, 2019.

This work was supported by the National Natural Science Foundation of China (Award No. 61732016). The content of this article does not necessarily reflect the views of this organization.

Media Relations Contact: Kimbra Cutlip, 301-405-9463, kcutlip@umd.edu
University of Maryland
College of Computer, Mathematical, and Natural Sciences
2300 Symons Hall
College Park, Md. 20742
http://www.cmns.umd.edu
@UMDscience

About the College of Computer, Mathematical, and Natural Sciences

The College of Computer, Mathematical, and Natural Sciences at the University of Maryland educates more than 9,000 future scientific leaders in its undergraduate and graduate programs each year. The college's 10 departments and more than a dozen interdisciplinary research centers foster scientific discovery with annual sponsored research funding exceeding $175 million.

University of Maryland

Related Data Articles:

Data centers use less energy than you think
Using the most detailed model to date of global data center energy use, researchers found that massive efficiency gains by data centers have kept energy use roughly flat over the past decade.
Storing data in music
Researchers at ETH Zurich have developed a technique for embedding data in music and transmitting it to a smartphone.
Life data economics: calling for new models to assess the value of human data
After the collapse of the blockchain bubble a number of research organisations are developing platforms to enable individual ownership of life data and establish the data valuation and pricing models.
Geoscience data group urges all scientific disciplines to make data open and accessible
Institutions, science funders, data repositories, publishers, researchers and scientific societies from all scientific disciplines must work together to ensure all scientific data are easy to find, access and use, according to a new commentary in Nature by members of the Enabling FAIR Data Steering Committee.
Democratizing data science
MIT researchers are hoping to advance the democratization of data science with a new tool for nonstatisticians that automatically generates models for analyzing raw data.
Getting the most out of atmospheric data analysis
An international team including researchers from Kanazawa University used a new approach to analyze an atmospheric data set spanning 18 years for the investigation of new-particle formation.
Ecologists ask: Should we be more transparent with data?
In a new Ecological Applications article, authors Stephen M. Powers and Stephanie E.
Should you share data of threatened species?
Scientists and conservationists have continually called for location data to be turned off in wildlife photos and publications to help preserve species but new research suggests there could be more to be gained by sharing a rare find, rather than obscuring it, in certain circumstances.
Futuristic data storage
The development of high-density data storage devices requires the highest possible density of elements in an array made up of individual nanomagnets.
Making data matter
The advent of 3-D printing has made it possible to take imaging data and print it into physical representations, but the process of doing so has been prohibitively time-intensive and costly.
More Data News and Data Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Uncharted
There's so much we've yet to explore–from outer space to the deep ocean to our own brains. This hour, Manoush goes on a journey through those uncharted places, led by TED Science Curator David Biello.
Now Playing: Science for the People

#556 The Power of Friendship
It's 2020 and times are tough. Maybe some of us are learning about social distancing the hard way. Maybe we just are all a little anxious. No matter what, we could probably use a friend. But what is a friend, exactly? And why do we need them so much? This week host Bethany Brookshire speaks with Lydia Denworth, author of the new book "Friendship: The Evolution, Biology, and Extraordinary Power of Life's Fundamental Bond". This episode is hosted by Bethany Brookshire, science writer from Science News.
Now Playing: Radiolab

Dispatch 2: Every Day is Ignaz Semmelweis Day
It began with a tweet: "EVERY DAY IS IGNAZ SEMMELWEIS DAY." Carl Zimmer – tweet author, acclaimed science writer and friend of the show – tells the story of a mysterious, deadly illness that struck 19th century Vienna, and the ill-fated hero who uncovered its cure ... and gave us our best weapon (so far) against the current global pandemic. This episode was reported and produced with help from Bethel Habte and Latif Nasser. Support Radiolab today at Radiolab.org/donate.