Nav: Home

How insights into human learning can foster smarter artificial intelligence

June 14, 2016

Recent breakthroughs in creating artificial systems that outplay humans in a diverse array of challenging games have their roots in neural networks inspired by information processing in the brain. In a Review published June 14 in Trends in Cognitive Sciences, researchers from Google DeepMind and Stanford University update a theory originally developed to explain how humans and other animals learn - and highlight its potential importance as a framework to guide the development of agents with artificial intelligence.

First published in 1995 (Psychol Rev., 102(3):419-57), the theory states that learning is the product of two complementary learning systems. The first system gradually acquires knowledge and skills from exposure to experiences, and the second stores specific experiences so that these can be replayed to allow their effective integration into the first system. The paper built on an earlier theory by influential British computational neuroscientist David Marr and on then-recent discoveries in neural network learning methods.

"The evidence seems compelling that the brain has these two kinds of learning systems, and the complementary learning systems theory explains how they complement each other to provide a powerful solution to a key learning problem that faces the brain," says Stanford Professor of Psychology James McClelland, lead author of the 1995 paper and senior author of the current Review.

The first system in the proposed theory, placed in the neocortex of the brain, was inspired by precursors of today's deep neural networks. As with today's deep networks, these systems contain several layers of neurons between input and output, and the knowledge in these networks is in their connections. Furthermore, their connections are gradually programmed by experience, giving rise to their ability to recognize objects, perceive speech, understand and produce language, and even to select optimal actions in game-playing and other settings where intelligent action depends on acquired knowledge.

Such systems face a dilemma when new information must be learned: If large enough changes are made to the connections to force the new knowledge into the connections quickly, it will radically distort all of the other knowledge already stored in the connections.

"That's where the complementary learning system comes in," McClelland says. In humans and other mammals, this second system is located in a structure called the hippocampus. "By initially storing information about the new experience in the hippocampus, we make it available for immediate use and we also keep it around so that it can be replayed back to the cortex, interleaving it with ongoing experience and stored information from other relevant experiences." This two-system set-up therefore allows both immediate learning and also gradual integration into the structured knowledge representation in the neocortex.

"Components of the neural network architecture that succeeded in achieving human-level performance in a variety of computer games like Space Invaders and Breakout were inspired by complementary learning systems theory" says DeepMind cognitive neuroscientist Dharshan Kumaran, the first author of the Review. "As in the theory, these neural networks exploit a memory buffer akin to the hippocampus that stores recent episodes of game play and replays them in interleaved fashion. This greatly amplifies the use of actual game play experience and avoids the tendency for a particular local run of experience to dominate learning in the system."

Kumaran has collaborated both with McClelland and with DeepMind co-founder Demis Hassabis (also a co-author on the Review), in work that extended the role of the hippocampus as it was envisioned in the 1995 version of the complementary learning systems theory.

"In my view," says Hassabis, "the extended version of the complementary learning systems theory is likely to continue to provide a framework for future research, not only in neuroscience but also in the quest to develop Artificial General Intelligence, our goal at Google DeepMind."
-end-
Trends in Cognitive Sciences, Kumaran et al.: "What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated" http://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(16)30043-2

Trends in Cognitive Sciences (@TrendsCognSci), published by Cell Press, is a monthly review journal that brings together research in psychology, artificial intelligence, linguistics, philosophy, computer science, and neuroscience. It provides a platform for the interaction of these disciplines and the evolution of cognitive science as an independent field of study. Visit: http://www.cell.com/trends/cognitive-sciences. To receive Cell Press media alerts, please contact press@cell.com.

Cell Press

Related Hippocampus Articles:

The hippocampus underlies the link between slowed walking and mental decline
The connection between slowed walking speed and declining mental acuity appears to arise in the right hippocampus, a finger-shaped region buried deep in the brain at ear-level, according to a 14-year study conducted by scientists at the University of Pittsburgh Graduate School of Public Health.
The brain's hippocampus can organize memories for events as well as places
Researchers at Japan's RIKEN Brain Science Institute have found that the hippocampus can generalize, putting not just places but also events into sequence by changing the neural code in the rat brain.
Plasma membrane protein may help generate new neurons in the adult hippocampus
New research published online in The FASEB Journal sheds important light on the inner workings of learning and memory.
Silent seizures recorded in the hippocampus of two patients with Alzheimer's disease
Massachusetts General Hospital investigators have identified silent, seizure-like activity in the hippocampus -- a brain structure significantly affected in Alzheimer's disease -- in two patients with Alzheimer's disease and no known history of seizures.
Brain tissue structure could explain link between fitness and memory
Studies have suggested a link between fitness and memory, but researchers have struggled to find the mechanism that links them.
Novel mode of antidepressant action may help patients unresponsive to SSRIs
Research at Osaka University identified a novel mode of action for a potential antidepressant that also leads to nerve cell growth in the mouse hippocampus.
Study identifies brain's connections which keep related memories distinct from each other
Neuroscientists at the University of Bristol are a step closer to understanding how the connections in our brain which control our episodic memory work in sync to make some memories stronger than others.
New system for forming memories
Until now, the hippocampus was considered the most important brain region for forming and recalling memory, with other regions only contributing as subordinates.
Moderate exercise improves memory dysfunction caused by type 2 diabetes
University of Tsukaba-led researchers showed that impaired glycometabolism and memory function in type 2 diabetic rats is improved by moderate exercise, possibly via enhanced lactate transport to neurons by MCT2.
Researchers uncover how hippocampus influences future thinking
Over the past decade, researchers have learned that the hippocampus -- historically known for its role in forming memories -- is involved in much more than just remembering the past; it plays an important role in imagining events in the future.

Related Hippocampus Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Changing The World
What does it take to change the world for the better? This hour, TED speakers explore ideas on activism—what motivates it, why it matters, and how each of us can make a difference. Guests include civil rights activist Ruby Sales, labor leader and civil rights activist Dolores Huerta, author Jeremy Heimans, "craftivist" Sarah Corbett, and designer and futurist Angela Oguntala.
Now Playing: Science for the People

#521 The Curious Life of Krill
Krill may be one of the most abundant forms of life on our planet... but it turns out we don't know that much about them. For a create that underpins a massive ocean ecosystem and lives in our oceans in massive numbers, they're surprisingly difficult to study. We sit down and shine some light on these underappreciated crustaceans with Stephen Nicol, Adjunct Professor at the University of Tasmania, Scientific Advisor to the Association of Responsible Krill Harvesting Companies, and author of the book "The Curious Life of Krill: A Conservation Story from the Bottom of the World".