Nav: Home

Method elucidates inner workings of neural networks

June 30, 2017

Neural networks, which learn to perform computational tasks by analyzing large sets of training data, are responsible for today's best-performing artificial intelligence systems, from speech recognition systems, to automatic translators, to self-driving cars.

But neural nets are black boxes. Once they've been trained, even their designers rarely have any idea what they're doing -- what data elements they're processing and how.

Two years ago, a team of computer-vision researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) described a method for peering into the black box of a neural net trained to identify visual scenes. The method provided some interesting insights, but it required data to be sent to human reviewers recruited through Amazon's Mechanical Turk crowdsourcing service.

At this year's Computer Vision and Pattern Recognition conference, CSAIL researchers will present a fully automated version of the same system. Where the previous paper reported the analysis of one type of neural network trained to perform one task, the new paper reports the analysis of four types of neural networks trained to perform more than 20 tasks, including recognizing scenes and objects, colorizing grey images, and solving puzzles. Some of the new networks are so large that analyzing any one of them would have been cost-prohibitive under the old method.

The researchers also conducted several sets of experiments on their networks that not only shed light on the nature of several computer-vision and computational-photography algorithms, but could also provide some evidence about the organization of the human brain.

Neural networks are so called because they loosely resemble the human nervous system, with large numbers of fairly simple but densely connected information-processing "nodes." Like neurons, a neural net's nodes receive information signals from their neighbors and then either "fire" -- emitting their own signals -- or don't. And as with neurons, the strength of a node's firing response can vary.

In both the new paper and the earlier one, the MIT researchers doctored neural networks trained to perform computer vision tasks so that they disclosed the strength with which individual nodes fired in response to different input images. Then they selected the 10 input images that provoked the strongest response from each node.

In the earlier paper, the researchers sent the images to workers recruited through Mechanical Turk, who were asked to identify what the images had in common. In the new paper, they use a computer system instead.

"We catalogued 1,100 visual concepts -- things like the color green, or a swirly texture, or wood material, or a human face, or a bicycle wheel, or a snowy mountaintop," says David Bau, an MIT graduate student in electrical engineering and computer science and one of the paper's two first authors. "We drew on several data sets that other people had developed, and merged them into a broadly and densely labeled data set of visual concepts. It's got many, many labels, and for each label we know which pixels in which image correspond to that label."

The paper's other authors are Bolei Zhou, co-first author and fellow graduate student; Antonio Torralba, MIT professor of electrical engineering and computer science; Aude Oliva, CSAIL principal research scientist; and Aditya Khosla, who earned his PhD as a member of Torralba's group and is now the chief technology officer of the medical-computing company PathAI.

The researchers also knew which pixels of which images corresponded to a given network node's strongest responses. Today's neural nets are organized into layers. Data are fed into the lowest layer, which processes them and passes them to the next layer, and so on. With visual data, the input images are broken into small chunks, and each chunk is fed to a separate input node.

For every strong response from a high-level node in one of their networks, the researchers could trace back the firing patterns that led to it, and thus identify the specific image pixels it was responding to. Because their system could frequently identify labels that corresponded to the precise pixel clusters that provoked a strong response from a given node, it could characterize the node's behavior with great specificity.

The researchers organized the visual concepts in their database into a hierarchy. Each level of the hierarchy incorporates concepts from the level below, beginning with colors and working upward through textures, materials, parts, objects, and scenes. Typically, lower layers of a neural network would fire in response to simpler visual properties -- such as colors and textures -- and higher layers would fire in response to more complex properties.

But the hierarchy also allowed the researchers to quantify the emphasis that networks trained to perform different tasks placed on different visual properties. For instance, a network trained to colorize black-and-white images devoted a large majority of its nodes to recognizing textures. Another network, when trained to track objects across several frames of video, devoted a higher percentage of its nodes to scene recognition than it did when trained to recognize scenes; in that case, many of its nodes were in fact dedicated to object detection.

One of the researchers' experiments could conceivably shed light on a vexed question in neuroscience. Research involving human subjects with electrodes implanted in their brains to control severe neurological disorders has seemed to suggest that individual neurons in the brain fire in response to specific visual stimuli. This hypothesis, originally called the grandmother-neuron hypothesis, is more familiar to a recent generation of neuroscientists as the Jennifer-Aniston-neuron hypothesis, after the discovery that several neurological patients had neurons that appeared to respond only to depictions of particular Hollywood celebrities.

Many neuroscientists dispute this interpretation. They argue that shifting constellations of neurons, rather than individual neurons, anchor sensory discriminations in the brain. Thus, the so-called Jennifer Aniston neuron is merely one of many neurons that collectively fire in response to images of Jennifer Aniston. And it's probably part of many other constellations that fire in response to stimuli that haven't been tested yet.

Because their new analytic technique is fully automated, the MIT researchers were able to test whether something similar takes place in a neural network trained to recognize visual scenes. In addition to identifying individual network nodes that were tuned to particular visual concepts, they also considered randomly selected combinations of nodes. Combinations of nodes, however, picked out far fewer visual concepts than individual nodes did -- roughly 80 percent fewer.

"To my eye, this is suggesting that neural networks are actually trying to approximate getting a grandmother neuron," Bau says. "They're not trying to just smear the idea of grandmother all over the place. They're trying to assign it to a neuron. It's this interesting hint of this structure that most people don't believe is that simple."
-end-
Additional background

Paper: Network dissection: Quantifying interpretability of deep visual representations

ARCHIVE: Explained: Neural networks

ARCHIVE: Computer learns to recognize sounds by watching video

ARCHIVE: Making computers explain themselves

ARCHIVE: Object recognition for free

Massachusetts Institute of Technology

Related Neurons Articles:

New tool to identify and control neurons
One of the big challenges in the Neuroscience field is to understand how connections and communications trigger our behavior.
Neurons that regenerate, neurons that die
In a new study published in Neuron, investigators report on a transcription factor that they have found that can help certain neurons regenerate, while simultaneously killing others.
How neurons use crowdsourcing to make decisions
When many individual neurons collect data, how do they reach a unanimous decision?
Neurons can learn temporal patterns
Individual neurons can learn not only single responses to a particular signal, but also a series of reactions at precisely timed intervals.
A turbo engine for tracing neurons
Putting a turbo engine into an old car gives it an entirely new life -- suddenly it can go further, faster.
Brain neurons help keep track of time
Turning the theory of how the human brain perceives time on its head, a novel analysis in mice reveals that dopamine neuron activity plays a key role in judgment of time, slowing down the internal clock.
During infancy, neurons are still finding their places
Researchers have identified a large population of previously unrecognized young neurons that migrate in the human brain during the first few months of life, contributing to the expansion of the frontal lobe, a region important for social behavior and executive function.
How many types of neurons are there in the brain?
For decades, scientists have struggled to develop a comprehensive census of cell types in the brain.
Molecular body guards for neurons
In the brain, patterns of neural activity are perfectly balanced.
Engineering researchers use laser to 'weld' neurons
University of Alberta researchers have developed a method of connecting neurons, using ultrashort laser pulses -- a breakthrough technique that opens the door to new medical research and treatment opportunities.

Related Neurons Reading:

The Neuron: Cell and Molecular Biology
by Irwin B. Levitan (Author), Leonard K. Kaczmarek (Author)

The Fourth Edition of The Neuron provides a comprehensive first course in the cell and molecular biology of nerve cells. The book begins with properties of the many newly discovered ion channels that have emerged through mapping of the genome. These channels shape the way a single neuron generates varied patterns of electrical activity. Covered next are the molecular mechanisms that convert electrical activity into the secretion of neurotransmitter hormones at synaptic junctions between neurons. The following section examines the biochemical pathways that are linked to the action of... View Details


The Neuron: Cell and Molecular Biology
by Irwin B. Levitan (Author), Leonard K. Kaczmarek (Author)

The third edition of The Neuron provides a comprehensive first course in the cell and molecular biology of nerve cells. The first part of the book covers the properties of the many ion channels that shape the way a single neuron generates varied patterns of electrical activity, as well as the molecular mechanisms that convert electrical activity into the secretion of neurotransmitter hormones at synaptic junctions between neurons. The second part covers the biochemical pathways that are linked to the action of neurotransmitters and can alter the cellular properties of neurons or... View Details


The 7 Secrets of Neuron Leadership: What Top Military Commanders, Neuroscientists, and the Ancient Greeks Teach Us about Inspiring Teams
by W. Craig Reed (Author), Gordon R. England (Foreword)

Leadership techniques backed by the world's most effective teams

The 7 Secrets of Neuron Leadership offers a diverse collection of wisdom and practical knowledge to help you build and lead your most effective team yet. Written by a former U.S. Navy diver, this book draws from the author's experiences and beyond to reveal key truths about the nature of teamwork, and expose the core of effective team leadership. You'll go back to ancient Greece to discover the nine personality types and the seven types of love that form the foundation of human interaction, and learn how... View Details


Molecular and Cellular Physiology of Neurons, Second Edition
by Gordon L. Fain (Author), Margery J. Fain (Illustrator), Thomas O'Dell (Illustrator)

Molecular and Cellular Physiology of Neurons, Second Edition is a comprehensive, up-to-date introduction to essential concepts of cellular neuroscience. Emphasizing experimental approaches and recent discoveries, it provides an in-depth look at the structure and function of nerve cells, from protein receptors and synapses to the biochemical processes that drive the mammalian nervous system.

Starting with the basics of electrical current flow across cell membranes, Gordon Fain covers voltage gating and receptor activation in the context of... View Details


From Neurons to Neighborhoods : The Science of Early Childhood Development
by Committee on Integrating the Science of Early Childhood Development (Author), Youth, and Families Board on Children (Author), National Research Council (Author), Committee on Integrating the Science of Early Childhood Development (Author), Jack P. Shonkoff (Editor), Deborah A. Phillips (Editor)

How we raise young children is one of today's most highly personalized and sharply politicized issues, in part because each of us can claim some level of "expertise." The debate has intensified as discoveries about our development-in the womb and in the first months and years-have reached the popular media.

How can we use our burgeoning knowledge to assure the well-being of all young children, for their own sake as well as for the sake of our nation? Drawing from new findings, this book presents important conclusions about nature-versus-nurture, the impact of being born into a... View Details


From Neuron to Brain
by John G. Nicholls (Author), A. Robert Martin (Author), David A. Brown (Author), Mathew E. Diamond (Author), David A. Weisblat (Author), Paul A. Fuchs (Author)

From Neuron to Brain, Fifth Edition, provides a readable, up-to-date book for use in undergraduate, graduate, and medical school courses in neuroscience. As in previous editions, the emphasis is on experiments made by electrical recordings, molecular and cellular biological techniques, and behavioral studies on the nervous system, from simple reflexes to cognitive functions. Lines of research are followed from the inception of an idea to new findings being made in laboratories and clinics today.

A major change is that this edition begins with the anatomy and physiology of the... View Details


From Photon to Neuron: Light, Imaging, Vision
by Philip Nelson (Author)

A richly illustrated undergraduate textbook on the physics and biology of light

Students in the physical and life sciences, and in engineering, need to know about the physics and biology of light. Recently, it has become increasingly clear that an understanding of the quantum nature of light is essential, both for the latest imaging technologies and to advance our knowledge of fundamental life processes, such as photosynthesis and human vision. From Photon to Neuron provides undergraduates with an accessible introduction to the physics of light and offers a unified view... View Details


Neuron: A Tutorial Study Guide
by Nicoladie Tam, Ph.D.

“Neuron” is a part of the college-level Principles of Biology course series and the Neuropsychopharmacology course series textbooks. It is a tutorial written in questions and answers format to describe the anatomy and physiology of the neurons in the brain.

It is a study guide with in-depth explanations. Each section is a modular unit that is self-contained for easy reading. The principles and concepts are introduced systematically so students can learn and retain the materials intuitively.
View Details


The Myth of Mirror Neurons: The Real Neuroscience of Communication and Cognition
by Gregory Hickok (Author)

An essential reconsideration of one of the most far-reaching theories in modern neuroscience and psychology.

In 1992, a group of neuroscientists from Parma, Italy, reported a new class of brain cells discovered in the motor cortex of the macaque monkey. These cells, later dubbed mirror neurons, responded equally well during the monkey’s own motor actions, such as grabbing an object, and while the monkey watched someone else perform similar motor actions. Researchers speculated that the neurons allowed the monkey to understand others by simulating their actions in its... View Details


From Neuron to Brain: A Cellular and Molecular Approach to the Function of the Nervous System, Fourth Edition
by John G. Nicholls (Author), A. Robert Martin (Author), Bruce G. Wallace (Author), Paul A. Fuchs (Author)

In the 25 years since From Neuron to Brain was first published, the authors' aim has remained constant—to describe how nerve cells go about their business of transmitting signals, how the signals are put together, and how, out of this integration, higher functions emerge. The new Fourth Edition, while maintaining this focus, has been completely reformatted and updated.

The emphasis, as before, is on experiments, and on the way they are carried out. Using a narrative approach, the authors follow a line from the original inception of a new idea to an account of research being done today.... View Details

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Turning Kids Into Grown-Ups
Parenting is fraught with uncertainty, changing with each generation. This hour, TED speakers share ideas about raising kids and how — despite our best efforts — we're probably still doing it wrong. Guests include former Stanford dean Julie Lythcott-Haims, former firefighter Caroline Paul, author Peggy Orenstein, psychologist Dr. Aala El-Khani, and poet Sarah Kay.
Now Playing: Science for the People

#470 Information Spookyhighway
This week we take a closer look at a few of the downsides of the modern internet, and some of the security and privacy challenges that are becoming increasingly troublesome. Rachelle Saunders speaks with cyber security expert James Lyne about how modern hacking differs from the hacks of old, and how an internet without national boards makes it tricky to police online crime across jurisdictions. And Bethany Brookshire speaks with David Garcia, a computer scientist at the Complexity Science Hub and the Medical University of Vienna, about the recent Cambridge Analytica scandal, and how social media platforms put a wrench...