Nav: Home

A system to synthesize realistic sounds for computer animation

August 06, 2018

Advances in computer-generated imagery have brought vivid, realistic animations to life, but the sounds associated with what we see simulated on screen, such as two objects colliding, are often recordings. Now researchers at Stanford University have developed a system that automatically renders accurate sounds for a wide variety of animated phenomena.

"There's been a Holy Grail in computing of being able to simulate reality for humans. We can animate scenes and render them visually with physics and computer graphics, but, as for sounds, they are usually made up," said Doug James, professor of computer science at Stanford University. "Currently there exists no way to generate realistic synchronized sounds for complex animated content, such as splashing water or colliding objects, automatically. This fills that void."

The researchers will present their work on this sound synthesis system as part of ACM SIGGRAPH 2018, the leading conference on computer graphics and interactive techniques. In addition to enlivening movies and virtual reality worlds, this system could also help engineering companies prototype how products would sound before being physically produced, and hopefully encourage designs that are quieter and less irritating, the researchers said.

"I've spent years trying to solve partial differential equations - which govern how sound propagates - by hand," said Jui-Hsien Wang, a graduate student in James' lab and in the Institute for Computational and Mathematical Engineering (ICME), and lead author of the paper. "This is actually a place where you don't just solve the equation but you can actually hear it once you've done it. That's really exciting to me and it's fun."

Predicting sound

Informed by geometry and physical motion, the system figures out the vibrations of each object and how, like a loudspeaker, those vibrations excite sound waves. It computes the pressure waves cast off by rapidly moving and vibrating surfaces but does not replicate room acoustics. So, although it does not recreate the echoes in a grand cathedral, it can resolve detailed sounds from scenarios like a crashing cymbal, an upside-down bowl spinning to a stop, a glass filling up with water or a virtual character talking into a megaphone.

Most sounds associated with animations rely on pre-recorded clips, which require vast manual effort to synchronize with the action on-screen. These clips are also restricted to noises that exist - they can't predict anything new. Other systems that produce and predict sounds as accurate as those of James and his team work only in special cases, or assume the geometry doesn't deform very much. They also require a long pre-computation phase for each separate object.

"Ours is essentially just a render button with minimal pre-processing that treats all objects together in one acoustic wave simulation," said Ante Qu, a graduate student in James' lab and co-author of the paper.

The simulated sound that results from this method is highly detailed. It takes into account the sound waves produced by each object in an animation but also predicts how those waves bend, bounce or deaden based on their interactions with other objects and sound waves in the scene.

Challenges ahead

In its current form, the group's process takes a while to create the finished product. But, now that they have proven this technique's potential, they can focus on performance optimizations, such as implementing their method on parallel GPU hardware, that should make it drastically faster.

And, even in its current state, the results are worth the wait.

"The first water sounds we generated with the system were among the best ones we had simulated - and water is a huge challenge in computer-generated sound," said James. "We thought we might get a little improvement, but it is dramatically better than previous approaches even right out of the box. It was really striking."

Although the group's work has faithfully rendered sounds of various objects spinning, falling and banging into each other, more complex objects and interactions - like the reverberating tones of a Stradivarius violin - remain difficult to model realistically. That, the group said, will have to wait for a future solution.
-end-
Timothy Langlois of Adobe Research is a co-author of this paper. This research was funded by the National Science Foundation and Adobe Research. James is also a professor, by courtesy, of music and a member of Stanford Bio-X.

Stanford University

Related Sound Waves Articles:

Sound waves direct particles to self-assemble, self-heal
Berkeley Lab scientists have demonstrated how floating particles will assemble and synchronize in response to acoustic waves.
Laser, sound waves provide live views of organs in action
Biomedical engineers are now able to take a live, holistic look at the inner workings of a small animal with enough resolution to see active organs, flowing blood, circulating melanoma cells and firing neural networks.
Sound waves boost older adult' memory, deep sleep
Gentle sound stimulation -- such as the rush of a waterfall -- synchronized to the rhythm of brain waves significantly enhanced deep sleep in older adults and tripled their ability to recall words, reports a new study.
Towards mastering terahertz waves?
Terahertz waves allow for the detection of materials that are undetectable at other frequencies.
Sound waves create whirlpools to round up tiny signs of disease
Mechanical engineers at Duke University have demonstrated a tiny whirlpool that can concentrate nanoparticles using nothing but sound.
The sound of quantum vacuum
Quantum mechanics dictates sensitivity limits in the measurements of displacement, velocity and acceleration.
The sound of a healthy reef
A new study from the Woods Hole Oceanographic Institution (WHOI) will help researchers understand the ways that marine animal larvae use sound as a cue to settle on coral reefs.
Knots in chaotic waves
New research, using computer models of wave chaos, has shown that three-dimensional tangled vortex filaments can in fact be knotted in many highly complex ways.
How to sound the alarm
A group of risk experts is proposing a new framework and research agenda that they believe will support the most effective public warnings when a hurricane, wildfire, toxic chemical spill or any other environmental hazard threatens safety.
Sound waves may hold potential to treat twin pregnancy complications
The early-stage feasibility study involving sheep suggests High Intensity Focused Ultrasound -- a technique already used for treating some cancers -- could help a condition called Twin-Twin Transfusion Syndrome.

Related Sound Waves Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Changing The World
What does it take to change the world for the better? This hour, TED speakers explore ideas on activism—what motivates it, why it matters, and how each of us can make a difference. Guests include civil rights activist Ruby Sales, labor leader and civil rights activist Dolores Huerta, author Jeremy Heimans, "craftivist" Sarah Corbett, and designer and futurist Angela Oguntala.
Now Playing: Science for the People

#521 The Curious Life of Krill
Krill may be one of the most abundant forms of life on our planet... but it turns out we don't know that much about them. For a create that underpins a massive ocean ecosystem and lives in our oceans in massive numbers, they're surprisingly difficult to study. We sit down and shine some light on these underappreciated crustaceans with Stephen Nicol, Adjunct Professor at the University of Tasmania, Scientific Advisor to the Association of Responsible Krill Harvesting Companies, and author of the book "The Curious Life of Krill: A Conservation Story from the Bottom of the World".