Nav: Home

Learning a scene that's unseen, without human help

June 14, 2018

Scientists at Google DeepMind have developed a machine-learning system that can "observe" a scene from multiple angles and predict what the same space would look from an entirely different view - one not encountered during training. The system learns the 3-D composition of an environment, the authors report, using only a small number of 2-D sample images of the scene - and, critically, without human supervision. Called the Generative Query Network (GQN), it could pave the way toward machines that can learn about the world autonomously using their own sensors, without the need for training with datasets labeled by humans, a requirement of current computer vision systems. Built by Seyed Mohammadali Eslami and colleagues, the GQN consists of two parts: a representation network, which develops an encoded representation of the scene from the sample images, and a generation network, which outputs probable images of the scene from new viewpoints, accounting for uncertainty when parts of the scene are obscured. Eslami and colleagues trained the GCN using simple computer-generated environments containing various objects and lighting setups. It could then be given several images of a new scene and was able to generate predicted images of that scene from any viewpoint within it. The network's representations are "factorized," meaning properties like color, shape and size are learned and used separately. The researchers were able to construct new scenes by adding or subtracting the GQN's representations together; subtracting a scene containing a red sphere from a scene with a blue sphere and adding one with a red cylinder results in a scene with a blue cylinder, all without a human explicitly teaching the GQN the notions of color or shape. The network also shows promise as a way to control robotic devices; after training, its predictive abilities allow it to "observe" robotic arms, for example, from different angles using only one stationary camera, meaning less raw data is needed for accurate positioning and control. A related perspective by Matthias Zwicker comments on these findings.

American Association for the Advancement of Science

Related Color Articles:

Why don't Americans have a name for the color 'light blue?'
'Mizu' translates to 'water' and has emerged in recent decades as a unique shade in the Japenese lexicon, new research has found.
The evolution of Japanese color vocabulary over the past 30 years
Color plays an important role in conveying visual information, but the language we use to express what we see is limited.
Jumping spiders court in color
UC biologist discovers unique visual diversities for rare color vision in two groups of jumping spiders.
A new technique for structural color, inspired by birds
Structural coloration has long interested researchers and engineers because of its durability and potential for application in solar arrays, biomimetic tissues and adaptive camouflage.
In food packaging, color matters
Food shoppers associate qualities of healthfulness and taste with the color of the packaging, so manufacturers must understand what light versus dark colors mean to buyers.
The color of birds
New research provides insight into plumage evolution.
UA report: Few studies look at well-being of LGB youth of color
While research on lesbian, gay and bisexual youth has increased, studies often fail to explore the well-being of young people of color.
What dinosaurs' color patterns say about their lives
After reconstructing the color patterns of a well-preserved dinosaur from China, researchers have found that the long-lost species called Psittacosaurus was light on its underside and darker on top.
Molecular switch for controlling color and fluorescence
Researchers in Japan have developed a molecular switching technique to control the visible color and fluorescent properties of a compound by using hydrogen and oxygen gas.
Camouflage artists, in color
A new study, co-authored by the father-and-son team of Christopher and Alexander Stubbs, suggests that chromatic aberration may explain how cephalopods -- the animal family that includes squid, octopi and cuttlefish -- can demonstrate such remarkable camouflage abilities despite only being able to see in black and white.

Related Color Reading:

The Secret Lives of Color
by Kassia St Clair (Author)

Color: A Natural History of the Palette
by Victoria Finlay (Author)

On Color
by David Kastan (Author), Stephen Farthing (Author)

Interaction of Color: 50th Anniversary Edition
by Josef Albers (Author), Nicholas Fox Weber (Foreword)

Secret Language of Color: Science, Nature, History, Culture, Beauty of Red, Orange, Yellow, Green, Blue, & Violet
by Joann Eckstut (Author), Arielle Eckstut (Author)

Color by Betty Edwards: A Course in Mastering the Art of Mixing Colors
by Betty Edwards (Author)

The Brilliant History of Color in Art
by Victoria Finlay (Author)

Color and Light: A Guide for the Realist Painter (James Gurney Art)
by James Gurney (Author)

Werner's Nomenclature of Colours: Adapted to Zoology, Botany, Chemistry, Mineralogy, Anatomy, and the Arts
by Patrick Syme (Author), Abraham Gottlob Werner (Illustrator)

Color, 2nd edition: A Workshop for Artists and Designers
by David Hornung (Author)

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Hacking The Law
We have a vision of justice as blind, impartial, and fair — but in reality, the law often fails those who need it most. This hour, TED speakers explore radical ways to change the legal system. Guests include lawyer and social justice advocate Robin Steinberg, animal rights lawyer Steven Wise, political activist Brett Hennig, and lawyer and social entrepreneur Vivek Maru.
Now Playing: Science for the People

#495 Earth Science in Space
Some worlds are made of sand. Some are made of water. Some are even made of salt. In science fiction and fantasy, planet can be made of whatever you want. But what does that mean for how the planets themselves work? When in doubt, throw an asteroid at it. This is a live show recorded at the 2018 Dragon Con in Atlanta Georgia. Featuring Travor Valle, Mika McKinnon, David Moscato, Scott Harris, and moderated by our own Bethany Brookshire. Note: The sound isn't as good as we'd hoped but we love the guests and the conversation and we wanted to...