Nav: Home

Learning a scene that's unseen, without human help

June 14, 2018

Scientists at Google DeepMind have developed a machine-learning system that can "observe" a scene from multiple angles and predict what the same space would look from an entirely different view - one not encountered during training. The system learns the 3-D composition of an environment, the authors report, using only a small number of 2-D sample images of the scene - and, critically, without human supervision. Called the Generative Query Network (GQN), it could pave the way toward machines that can learn about the world autonomously using their own sensors, without the need for training with datasets labeled by humans, a requirement of current computer vision systems. Built by Seyed Mohammadali Eslami and colleagues, the GQN consists of two parts: a representation network, which develops an encoded representation of the scene from the sample images, and a generation network, which outputs probable images of the scene from new viewpoints, accounting for uncertainty when parts of the scene are obscured. Eslami and colleagues trained the GCN using simple computer-generated environments containing various objects and lighting setups. It could then be given several images of a new scene and was able to generate predicted images of that scene from any viewpoint within it. The network's representations are "factorized," meaning properties like color, shape and size are learned and used separately. The researchers were able to construct new scenes by adding or subtracting the GQN's representations together; subtracting a scene containing a red sphere from a scene with a blue sphere and adding one with a red cylinder results in a scene with a blue cylinder, all without a human explicitly teaching the GQN the notions of color or shape. The network also shows promise as a way to control robotic devices; after training, its predictive abilities allow it to "observe" robotic arms, for example, from different angles using only one stationary camera, meaning less raw data is needed for accurate positioning and control. A related perspective by Matthias Zwicker comments on these findings.
-end-


American Association for the Advancement of Science

Related Color Articles:

Why don't Americans have a name for the color 'light blue?'
'Mizu' translates to 'water' and has emerged in recent decades as a unique shade in the Japenese lexicon, new research has found.
The evolution of Japanese color vocabulary over the past 30 years
Color plays an important role in conveying visual information, but the language we use to express what we see is limited.
Jumping spiders court in color
UC biologist discovers unique visual diversities for rare color vision in two groups of jumping spiders.
A new technique for structural color, inspired by birds
Structural coloration has long interested researchers and engineers because of its durability and potential for application in solar arrays, biomimetic tissues and adaptive camouflage.
In food packaging, color matters
Food shoppers associate qualities of healthfulness and taste with the color of the packaging, so manufacturers must understand what light versus dark colors mean to buyers.
The color of birds
New research provides insight into plumage evolution.
UA report: Few studies look at well-being of LGB youth of color
While research on lesbian, gay and bisexual youth has increased, studies often fail to explore the well-being of young people of color.
What dinosaurs' color patterns say about their lives
After reconstructing the color patterns of a well-preserved dinosaur from China, researchers have found that the long-lost species called Psittacosaurus was light on its underside and darker on top.
Molecular switch for controlling color and fluorescence
Researchers in Japan have developed a molecular switching technique to control the visible color and fluorescent properties of a compound by using hydrogen and oxygen gas.
Camouflage artists, in color
A new study, co-authored by the father-and-son team of Christopher and Alexander Stubbs, suggests that chromatic aberration may explain how cephalopods -- the animal family that includes squid, octopi and cuttlefish -- can demonstrate such remarkable camouflage abilities despite only being able to see in black and white.

Related Color Reading:

The Secret Lives of Color
by Kassia St Clair (Author)

The unforgettable, unknown history of colors and the vivid stories behind them in a beautiful multi-colored volume

“Beautifully written . . . Full of anecdotes and fascinating research, this elegant compendium has all the answers.” —NPR, Best Books of 2017

The Secret Lives of Color tells the unusual stories of seventy-five fascinating shades, dyes and hues. From blonde to ginger, the brown that changed the way battles were fought to the white that protected against the plague, Picasso’s blue period to the charcoal on the cave walls at... View Details


On Color
by David Kastan (Author), Stephen Farthing (Author)

Our lives are saturated by color. We live in a world of vivid colors, and color marks our psychological and social existence. But for all color’s inescapability, we don’t know much about it. Now authors David Scott Kastan and Stephen Farthing offer a fresh and imaginative exploration of one of the most intriguing and least understood aspects of everyday experience.

Kastan and Farthing, a scholar and a painter, respectively, investigate color from numerous perspectives: literary, historical, cultural, anthropological, philosophical, art historical, political, and scientific. In... View Details


Color A Natural History of the Palette
by Victoria Finlay (Author)

Discover the tantalizing true stories behind your favorite colors.
For example: Cleopatra used saffron—a source of the color yellow—for seduction. Extracted from an Afghan mine, the blue “ultramarine” paint used by Michelangelo was so expensive he couldn’t afford to buy it himself. Since ancient times, carmine red—still found in lipsticks and Cherry Coke today—has come from the blood of insects. View Details


Interaction of Color 50th Anniversary Edition
by Josef Albers (Author), Nicholas Fox Weber (Foreword)

Josef Albers’s Interaction of Color is a masterwork in art education. Conceived as a handbook and teaching aid for artists, instructors, and students, this influential book presents Albers’s singular explanation of complex color theory principles.
 
Originally published by Yale University Press in 1963 as a limited silkscreen edition with 150 color plates, Interaction of Color first appeared in paperback in 1971, featuring ten color studies chosen by Albers, and has remained in print ever since. With over a quarter of a million copies sold in its various... View Details


Werner's Nomenclature of Colours: Adapted to Zoology, Botany, Chemistry, Mineralogy, Anatomy, and the Arts
by Patrick Syme (Author), Abraham Gottlob Werner (Illustrator)

First published in 1814, Werner's Nomenclature of Colours is a taxonomic guide to the colors of the natural world that has been cherished by artists and scientists for more than two centuries. This beautiful pocket-size facsimile is certain to delight and inform a new generation of artists and scientists. Werner's Nomenclature of Colours is a charming artifact from the golden age of natural history and global exploration.

In the late eighteenth century, mineralogist Abraham Gottlob Werner devised a standardized color scheme that allowed him to describe even the... View Details


Color Theory: An essential guide to color-from basic principles to practical applications (Artist's Library)
by Patti Mollica (Author)

Color Theory features everything an artist might need to know about color, including color psychology, pigment characteristics and terms, color schemes, color mixing, shadows, highlights, and much more. Regardless of your medium, a solid understanding of color and its applications is essential. With a fresh, contemporary take on the subject of color, this essential guide features step-by-step projects and practical tips and techniques to put color knowledge to effective use.

View Details


Find Colors: Published in association with the Whitney Museum of American Art
by Tamara Shopsin Jason Fulford (Author)

A colors book with no colors ? instead, die-cuts to peek through and find them yourself

This introduction to colors integrates the reader's surroundings into carefully considered die-cut silhouettes, providing children the space to visually experiment. Readers will gaze around the room through a rooster-shaped hole in search of something red, through a sun-shaped hole for something yellow, through squiggly worms for something pink. Designed for the youngest readers, this sturdy board book features 12 die-cuts made to flip and carry on a color-seeking mission.... View Details


Color by Betty Edwards: A Course in Mastering the Art of Mixing Colors
by Betty Edwards (Author)

Millions of people have learned to draw using the methods of Dr. Betty Edwards's bestseller The New Drawing on the Right Side of the Brain. Now, much as artists progress from drawing to painting, Edwards moves from black-and-white into color. This much-awaited new guide distills the enormous existing knowledge about color theory into a practical method of working with color to produce harmonious combinations.

Using techniques tested and honed in her five-day intensive color workshops, Edwards provides a basic understanding of how to see color, how to use it, and-for those... View Details


The Designer's Dictionary of Color
by Sean Adams (Author)

The Designer's Dictionary of Color View Details


The Natural Colors Cookbook: Custom Hues For Your Fabrics Made Simple Using Food
by Maggie Pate (Author)

The Essential Manual for Creating Custom Natural Dyes at Home

Discover the wonder of nature’s living colors and transform everyday items with The Natural Colors Cookbook, your guide to creating a spectrum of organic dyes using seasonal produce and leftover food. Now you can bring the timeless form of plant-based dyes to the comfort and familiarity of your own kitchen with the expertise of Maggie Pate, owner and designer of the naturally-dyed clothing label Nåde. Watch in awe as a few hours on the stove extracts a delicate blush tone from a handful of avocado pits,... View Details

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

The Person You Become
Over the course of our lives, we shed parts of our old selves, embrace new ones, and redefine who we are. This hour, TED speakers explore ideas about the experiences that shape the person we become. Guests include aerobatics pilot and public speaker Janine Shepherd, writers Roxane Gay and Taiye Selasi, activist Jackson Bird, and fashion executive Kaustav Dey.
Now Playing: Science for the People

#479 Garden of Marvels (Rebroadcast)
This week we're learning about botany and the colorful science of gardening. Author Ruth Kassinger joins us to discuss her book "A Garden of Marvels: How We Discovered that Flowers Have Sex, Leaves Eat Air, and Other Secrets of the Way Plants Work." And we'll speak to NASA researcher Gioia Massa about her work to solve the technical challenges of gardening in space.