Nav: Home

Green-screen keying method cuts time, boosts quality in film compositing

November 14, 2016

Filming an actor in front of a green screen and then superimposing the actor over another background is commonplace in feature film production, but getting rid of all traces of the green screen remains a chore. A new "keying" method devised by Disney Research and ETH Zurich, however, improves the results substantially.

What's more, these superior results can be achieved in about one-tenth the time it takes a professional compositor to perform this task manually.

"Keying is a crucial intermediate step in adding synthetic backgrounds and objects seamlessly into live-action shots," said Markus Gross, vice president at Disney Research. "It's also a major bottleneck in postproduction, so a keying method that is both fast and produces high-quality results will have a major impact on the production of feature films."

A report on the method, which is based on a novel algorithm that computes underlying colors and their mixing ratios, will be presented at the Association for Computing Machinery's (ACM) Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2017) next summer in Los Angeles and has been published in the August issue of ACM Transactions on Graphics journal (ACM TOG).

A number of commercial software tools are available to help a compositing artist perform keying, said Tunç Aydin, research scientist at Disney Research. But those tools have difficulty discerning the green screen background from the foreground in regions of the image where the colors of objects mix with each other, he noted. This mixing can occur because of motion blur, intricate object boundaries, such as hair, or colors cast onto an object from the green screen itself. Whatever the cause, these imperfections usually force the artist to painstakingly paint those regions manually.

The research team opted to create a color unmixing algorithm, which can be used to extract multiple color layers using a color model of the image, said Yagiz Aksoy, a Ph.D. student at Disney Research and ETH Zurich. A subset of these colors is then combined to produce the final keying result.

The team created a user interface that enables the artist to create a color model of the image simply by scribbling over each distinct-colored area of the image. The shape of the scribble doesn't matter; the program simply needs to identify each area of color so it can create a representation of all the colors in the scene. The algorithm computes the underlying colors and their mixing ratios for each area.

The researchers showed that the method would be applicable for keying scenes with non-green-screen backgrounds. Also, the ability to represent the image with multiple color layers, rather than just foreground and background layers as in conventional methods, creates opportunities for new applications, Aydin said. Colors of objects in a scene, for instance, could be changed.

In comprehensive comparisons with commercial keying tools and with other published methods, the researchers showed that the results for their method were superior in quality as it required only one-tenth of the manual editing time it would take a professional artist to process the same content.

Combining creativity and innovation, this research continues Disney's rich legacy of inventing new ways to tell great stories and leveraging technology required to build the future of entertainment.
-end-
In addition to Aksoy and Aydin, the research team included Marc Pollefeys, professor of computer science at ETH Zurich, and Aljoša Smolic, a former senior research scientist at Disney Research, currently a professor at Trinity College in Dublin.

For more information and a video, visit the project web site at https://www.disneyresearch.com/publication/green-screen/.

About Disney Research

Disney Research is a network of research laboratories supporting The Walt Disney Company. Its purpose is to pursue scientific and technological innovation to advance the company's broad media and entertainment efforts. Vice Presidents Jessica Hodgins and Markus Gross manage Disney Research facilities in Los Angeles, Pittsburgh and Zürich, and work closely with the Pixar and ILM research groups in the San Francisco Bay Area. Research topics include computer graphics, animation, video processing, computer vision, robotics, wireless & mobile computing, human-computer interaction, displays, behavioral economics, and machine learning.

Website: http://www.disneyresearch.com
Twitter: @DisneyResearch
Facebook: http://www.facebook.com/DisneyResearch

Disney Research

Related Algorithm Articles:

New algorithm could unleash the power of quantum computers
A new algorithm that fast forwards simulations could bring greater use ability to current and near-term quantum computers, opening the way for applications to run past strict time limits that hamper many quantum calculations.
QUT algorithm could quash Twitter abuse of women
Online abuse targeting women, including threats of harm or sexual violence, has proliferated across all social media platforms but QUT researchers have developed a sophisticated statistical model to identify misogynistic content and help drum it out of the Twittersphere.
New learning algorithm should significantly expand the possible applications of AI
The e-prop learning method developed at Graz University of Technology forms the basis for drastically more energy-efficient hardware implementations of Artificial Intelligence.
Algorithm predicts risk for PTSD after traumatic injury
With high precision, a new algorithm predicts which patients treated for traumatic injuries in the emergency department will later develop posttraumatic stress disorder.
New algorithm uses artificial intelligence to help manage type 1 diabetes
Researchers and physicians at Oregon Health & Science University have designed a method to help people with type 1 diabetes better manage their glucose levels.
A new algorithm predicts the difficulty in fighting fire
The tool completes previous studies with new variables and could improve the ability to respond to forest fires.
New algorithm predicts optimal materials among all possible compounds
Skoltech researchers have offered a solution to the problem of searching for materials with required properties among all possible combinations of chemical elements.
New algorithm to help process biological images
Skoltech researchers have presented a new biological image processing method that accurately picks out specific biological objects in complex images.
Skoltech scientists break Google's quantum algorithm
In the near term, Google has devised new quantum enhanced algorithms that operate in the presence of realistic noise.
The most human algorithm
A team from the research group SEES:lab of the Department of Chemical Engineering of the Universitat Rovira I Virgili and ICREA has made a breakthrough with the development of a new algorithm that makes more accurate predictions and generates mathematical models that also make it possible to understand these predictions.
More Algorithm News and Algorithm Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Power Of Spaces
How do spaces shape the human experience? In what ways do our rooms, homes, and buildings give us meaning and purpose? This hour, TED speakers explore the power of the spaces we make and inhabit. Guests include architect Michael Murphy, musician David Byrne, artist Es Devlin, and architect Siamak Hariri.
Now Playing: Science for the People

#576 Science Communication in Creative Places
When you think of science communication, you might think of TED talks or museum talks or video talks, or... people giving lectures. It's a lot of people talking. But there's more to sci comm than that. This week host Bethany Brookshire talks to three people who have looked at science communication in places you might not expect it. We'll speak with Mauna Dasari, a graduate student at Notre Dame, about making mammals into a March Madness match. We'll talk with Sarah Garner, director of the Pathologists Assistant Program at Tulane University School of Medicine, who takes pathology instruction out of...
Now Playing: Radiolab

What If?
There's plenty of speculation about what Donald Trump might do in the wake of the election. Would he dispute the results if he loses? Would he simply refuse to leave office, or even try to use the military to maintain control? Last summer, Rosa Brooks got together a team of experts and political operatives from both sides of the aisle to ask a slightly different question. Rather than arguing about whether he'd do those things, they dug into what exactly would happen if he did. Part war game part choose your own adventure, Rosa's Transition Integrity Project doesn't give us any predictions, and it isn't a referendum on Trump. Instead, it's a deeply illuminating stress test on our laws, our institutions, and on the commitment to democracy written into the constitution. This episode was reported by Bethel Habte, with help from Tracie Hunte, and produced by Bethel Habte. Jeremy Bloom provided original music. Support Radiolab by becoming a member today at Radiolab.org/donate.     You can read The Transition Integrity Project's report here.