Disney Research's interactive method synchronizes multiple videos

August 08, 2014

Disney Research Zurich has developed a new tool to help video editors synchronize multiple video clips based on the visual content of the videos, rather than relying on timecodes or other external markers. Current editing tools include a "snapping" interface that aligns video clips based on start-and-end times; by contrast, Disney Research's VideoSnapping method is based on an analysis of the content of the video. This makes it easier to synchronize multiple clips without such cues as global timecodes or audio, and even when clips are shot at different trajectories and different formats.

"Being able to synchronize multiple videos based only on visual similarities makes possible some exciting new applications," said Oliver Wang, research scientist at DRZ. "For instance, popular events today increasingly are recorded by a large number of cameras, such as smartphones; with VideoSnapping, we can efficiently create a single video from these crowdsourced video clips, even though they are shot from different positions on devices with different capture rates and may have little or no overlap between them."

Wang and the rest of the Disney Research Zurich team will present their findings at ACM SIGGRAPH 2014, the International Conference on Computer Graphics and Interactive Techniques, Aug. 10-14, in Vancouver, Canada.

"Our video alignment technology was originally inspired by an experimental short film we produced last year in Switzerland. In one shot we had to combine multiple takes of a scene from hand-guided cameras and there was no obvious way to do this easily," added Markus Gross, the director of Disney Research in Zurich and the film's producer.

Alignment of video clips is a task essential to a number of applications, including the compositing of multiple takes in movie production and visual effects, stitching together of video mosaics and combining multiple videos at different exposure levels to create high-dynamic range (HDR) videos.

Without timecodes or audio to guide temporal alignment of clips, editors now must manually align them, usually only with the help of start and endpoints or pre-specified markers. By contrast, with the VideoSnapping method, an editor has the option to drag a pair of video clips in a timeline-style interface, where an algorithm analyzes features in the video frames and determines the best match, causing the videos to snap into alignment. Alternately, temporal synchronization can be automatically computed, allowing for the method to scale to large numbers of clips without need for interaction.

"A major benefit of our temporal snapping is that it can greatly reduce the difficulty of finding good spatial alignments," said Alexander Sorkine-Hornung, Disney Research Zurich senior research scientist. "In fact, we have shown that by applying a simple, out-of-the-box method for spatial alignment after synchronization, we can achieve similar or better quality when compared to state-of-the-art spatiotemporal approaches."

This method doesn't work if the videos lack usable common features. But the researchers have identified possible workarounds. For instance, in a case where videos show two different people repeating a series of actions, synchronization is possible by using skeleton data extracted from Kinect sensors.
In addition to Sorkine-Hornung and Wang, the research team included Disney Research Zurich's Christopher Schroers and Henning Zimmer and DRZ director Markus Gross. For more information on the project and explanatory video, visit the web site at http://www.disneyresearch.com/project/videosnapping.

About Disney Research

Disney Research is a network of research laboratories supporting The Walt Disney Company. Its purpose is to pursue scientific and technological innovation to advance the company's broad media and entertainment efforts. Vice Presidents Jessica Hodgins and Markus Gross manage Disney Research facilities in Los Angeles, Pittsburgh, Zürich, and Boston and work closely with the Pixar and ILM research groups in the San Francisco Bay Area. Research topics include computer graphics, animation, video processing, computer vision, robotics, wireless & mobile computing, human-computer interaction, displays, behavioral economics, and machine learning.

Disney Research

Related Synchronization Articles from Brightsurf:

Study: Most migratory birds rely on a greening world
A new study from the Cornell Lab of Ornithology confirms that most birds -- but not all -- synchronize their migratory movements with seasonal changes in vegetation greenness.

Safe flight: New method detects onset of destructive oscillations in aircraft turbines
''Flutter'' is a complex oscillatory phenomenon that can destroy aircraft turbine blades and has historically been the cause of several plane accidents.

What violin synchronization can teach us about better networking in complex times
A new study published in Nature Communications suggests by using a model of violin synchronization in a network of violin players, there are ways to drown out distractions and miscommunications that could be used as a model for human networks in society.

Network of sounds: New research reveals the magic secret of human networks
A group of Israeli researchers recruited 16 violinists to study the behavior of a human network and find out what sets it apart from other networks, such as animals, computers and other objects.

Fighting fish synchronize their combat moves and their gene expression
When two betta fish are fighting for dominance, not only do their attacks mirror each other, but the gene expression in their brain cells also starts to align.

Topology sheds new light on synchronization in higher-order networks
Research led by Queen Mary University of London, proposes a novel 'higher-order' Kuramoto model that combines topology with dynamical systems and characterises synchronization in higher-order networks for the first time.

Hearts that drum together beat together
Researchers from Bar-Ilan University found that in a structured group drumming task aspects of participants' heart function synchronized.

Many buds to a blossom: A synchronization approach to sensing using many oscillators
Engineers at the Tokyo Tech have found a new approach of taking a measurement over an extended area.

Walking together: Personal traits and first impressions affects step synchronization
Walking is one of our most natural, daily actions. Now, a new study led by a Tohoku University researcher suggests that walkers use step synchronization as a form of non-verbal social communication.

UNH researchers find synchronization of memory cells critical for learning and forming memories
Researchers at the University of New Hampshire found that the neurons, or memory cells, involved in Pavlovian learning shift their behavior during the learning process and become more synchronized when a memory is formed -- a finding that helps better understand memory mechanisms and provides clues for the development of future therapies for memory-related diseases like dementia, autism and post-traumatic stress disorder (PTSD).

Read More: Synchronization News and Synchronization Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.