Nav: Home

Learning a scene that's unseen, without human help

June 14, 2018

Scientists at Google DeepMind have developed a machine-learning system that can "observe" a scene from multiple angles and predict what the same space would look from an entirely different view - one not encountered during training. The system learns the 3-D composition of an environment, the authors report, using only a small number of 2-D sample images of the scene - and, critically, without human supervision. Called the Generative Query Network (GQN), it could pave the way toward machines that can learn about the world autonomously using their own sensors, without the need for training with datasets labeled by humans, a requirement of current computer vision systems. Built by Seyed Mohammadali Eslami and colleagues, the GQN consists of two parts: a representation network, which develops an encoded representation of the scene from the sample images, and a generation network, which outputs probable images of the scene from new viewpoints, accounting for uncertainty when parts of the scene are obscured. Eslami and colleagues trained the GCN using simple computer-generated environments containing various objects and lighting setups. It could then be given several images of a new scene and was able to generate predicted images of that scene from any viewpoint within it. The network's representations are "factorized," meaning properties like color, shape and size are learned and used separately. The researchers were able to construct new scenes by adding or subtracting the GQN's representations together; subtracting a scene containing a red sphere from a scene with a blue sphere and adding one with a red cylinder results in a scene with a blue cylinder, all without a human explicitly teaching the GQN the notions of color or shape. The network also shows promise as a way to control robotic devices; after training, its predictive abilities allow it to "observe" robotic arms, for example, from different angles using only one stationary camera, meaning less raw data is needed for accurate positioning and control. A related perspective by Matthias Zwicker comments on these findings.

American Association for the Advancement of Science

Related Color Articles:

Why don't Americans have a name for the color 'light blue?'
'Mizu' translates to 'water' and has emerged in recent decades as a unique shade in the Japenese lexicon, new research has found.
The evolution of Japanese color vocabulary over the past 30 years
Color plays an important role in conveying visual information, but the language we use to express what we see is limited.
Jumping spiders court in color
UC biologist discovers unique visual diversities for rare color vision in two groups of jumping spiders.
A new technique for structural color, inspired by birds
Structural coloration has long interested researchers and engineers because of its durability and potential for application in solar arrays, biomimetic tissues and adaptive camouflage.
In food packaging, color matters
Food shoppers associate qualities of healthfulness and taste with the color of the packaging, so manufacturers must understand what light versus dark colors mean to buyers.
The color of birds
New research provides insight into plumage evolution.
UA report: Few studies look at well-being of LGB youth of color
While research on lesbian, gay and bisexual youth has increased, studies often fail to explore the well-being of young people of color.
What dinosaurs' color patterns say about their lives
After reconstructing the color patterns of a well-preserved dinosaur from China, researchers have found that the long-lost species called Psittacosaurus was light on its underside and darker on top.
Molecular switch for controlling color and fluorescence
Researchers in Japan have developed a molecular switching technique to control the visible color and fluorescent properties of a compound by using hydrogen and oxygen gas.
Camouflage artists, in color
A new study, co-authored by the father-and-son team of Christopher and Alexander Stubbs, suggests that chromatic aberration may explain how cephalopods -- the animal family that includes squid, octopi and cuttlefish -- can demonstrate such remarkable camouflage abilities despite only being able to see in black and white.

Related Color Reading:

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

The Right To Speak
Should all speech, even the most offensive, be allowed on college campuses? And is hearing from those we deeply disagree with ... worth it? This hour, TED speakers explore the debate over free speech. Guests include recent college graduate Zachary Wood, political scientist Jeffrey Howard, novelist Elif Shafak, and journalist and author James Kirchick.
Now Playing: Science for the People

#486 Volcanoes
This week we're talking volcanoes. Because there are few things that fascinate us more than the amazing, unstoppable power of an erupting volcano. First, Jessica Johnson takes us through the latest activity from the Kilauea volcano in Hawaii to help us understand what's happening with this headline-grabbing volcano. And Janine Krippner joins us to highlight some of the lesser-known volcanoes that can be found in the USA, the different kinds of eruptions we might one day see at them, and how damaging they have the potential to be. Related links: Kilauea status report at USGS A beginner's guide to Hawaii's otherworldly...