Nav: Home

Learning a scene that's unseen, without human help

June 14, 2018

Scientists at Google DeepMind have developed a machine-learning system that can "observe" a scene from multiple angles and predict what the same space would look from an entirely different view - one not encountered during training. The system learns the 3-D composition of an environment, the authors report, using only a small number of 2-D sample images of the scene - and, critically, without human supervision. Called the Generative Query Network (GQN), it could pave the way toward machines that can learn about the world autonomously using their own sensors, without the need for training with datasets labeled by humans, a requirement of current computer vision systems. Built by Seyed Mohammadali Eslami and colleagues, the GQN consists of two parts: a representation network, which develops an encoded representation of the scene from the sample images, and a generation network, which outputs probable images of the scene from new viewpoints, accounting for uncertainty when parts of the scene are obscured. Eslami and colleagues trained the GCN using simple computer-generated environments containing various objects and lighting setups. It could then be given several images of a new scene and was able to generate predicted images of that scene from any viewpoint within it. The network's representations are "factorized," meaning properties like color, shape and size are learned and used separately. The researchers were able to construct new scenes by adding or subtracting the GQN's representations together; subtracting a scene containing a red sphere from a scene with a blue sphere and adding one with a red cylinder results in a scene with a blue cylinder, all without a human explicitly teaching the GQN the notions of color or shape. The network also shows promise as a way to control robotic devices; after training, its predictive abilities allow it to "observe" robotic arms, for example, from different angles using only one stationary camera, meaning less raw data is needed for accurate positioning and control. A related perspective by Matthias Zwicker comments on these findings.

American Association for the Advancement of Science

Related Color Articles:

Why don't Americans have a name for the color 'light blue?'
'Mizu' translates to 'water' and has emerged in recent decades as a unique shade in the Japenese lexicon, new research has found.
The evolution of Japanese color vocabulary over the past 30 years
Color plays an important role in conveying visual information, but the language we use to express what we see is limited.
Jumping spiders court in color
UC biologist discovers unique visual diversities for rare color vision in two groups of jumping spiders.
A new technique for structural color, inspired by birds
Structural coloration has long interested researchers and engineers because of its durability and potential for application in solar arrays, biomimetic tissues and adaptive camouflage.
In food packaging, color matters
Food shoppers associate qualities of healthfulness and taste with the color of the packaging, so manufacturers must understand what light versus dark colors mean to buyers.
The color of birds
New research provides insight into plumage evolution.
UA report: Few studies look at well-being of LGB youth of color
While research on lesbian, gay and bisexual youth has increased, studies often fail to explore the well-being of young people of color.
What dinosaurs' color patterns say about their lives
After reconstructing the color patterns of a well-preserved dinosaur from China, researchers have found that the long-lost species called Psittacosaurus was light on its underside and darker on top.
Molecular switch for controlling color and fluorescence
Researchers in Japan have developed a molecular switching technique to control the visible color and fluorescent properties of a compound by using hydrogen and oxygen gas.
Camouflage artists, in color
A new study, co-authored by the father-and-son team of Christopher and Alexander Stubbs, suggests that chromatic aberration may explain how cephalopods -- the animal family that includes squid, octopi and cuttlefish -- can demonstrate such remarkable camouflage abilities despite only being able to see in black and white.

Related Color Reading:

The Secret Lives of Color
by Kassia St Clair (Author)

On Color
by David Kastan (Author), Stephen Farthing (Author)

Color: A Natural History of the Palette
by Victoria Finlay (Author)

Interaction of Color: 50th Anniversary Edition
by Josef Albers (Author), Nicholas Fox Weber (Foreword)

Secret Language of Color: Science, Nature, History, Culture, Beauty of Red, Orange, Yellow, Green, Blue, & Violet
by Joann Eckstut (Author), Arielle Eckstut (Author)

Lean Yoga: Shed The Excess Weight From Your Body And Mind (Yoga for Weight Loss, Stress, Anxiety Relief, Mindfulness and Inner Peace, Includes Color Photos of all Asanas)

Color by Betty Edwards: A Course in Mastering the Art of Mixing Colors
by Betty Edwards (Author)

A Dictionary Of Color Combinations
by Various (Author)

Fitzpatrick's Color Atlas, 8th Ed
by Klaus Wolff (Author), Richard Allen Johnson (Author), Arturo P. Saavedra (Author), Ellen K. Roh (Author)

Color, 2nd edition: A Workshop for Artists and Designers
by David Hornung (Author)

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Approaching With Kindness
We often forget to say the words "thank you." But can those two words change how you — and those around you — look at the world? This hour, TED speakers on the power of gratitude and appreciation. Guests include author AJ Jacobs, author and former baseball player Mike Robbins, Dr. Laura Trice, Professor of Management Christine Porath, and former Danish politician Özlem Cekic.
Now Playing: Science for the People

#509 Anisogamy: The Beginning of Male and Female
This week we discuss how the sperm and egg came to be, and how a difference of reproductive interest has led to sexual conflict in bed bugs. We'll be speaking with Dr. Geoff Parker, an evolutionary biologist credited with developing a theory to explain the evolution of two sexes, about anisogamy, sexual reproduction through the fusion of two different gametes: the egg and the sperm. Then we'll speak with Dr. Roberto Pereira, research scientist in urban entomology at the University of Florida, about traumatic insemination in bed bugs.