Nav: Home

Dopamine neurons factor ambiguity into predictions enabling us to 'win big and win often'

March 09, 2017

Cold Spring Harbor, NY - In the struggle of life, evolution rewards animals that master their circumstances, especially when the environment changes fast. If there is a recipe for success, it is not: savor your victories when you are fortunate to have them. Rather it is: win big, and win often.

To make winning decisions, animals cannot consult a hard-wired playbook or receive helpful advice. Instead, success depends on their ability to learn, from triumphs and from mistakes.

Today in the journal Current Biology, a team led by Professor Adam Kepecs of Cold Spring Harbor Laboratory (CSHL) adds an important new dimension to our understanding of how the brain learns, focusing specifically on situations in which perceptually ambiguous information is used to make predictions.

What if it's gotten foggy and we can barely see the sign up ahead, or the announcement over the loudspeaker is garbled? In circumstances like these, how does the brain adjust the predictions that inform decisions?

Kepecs and colleagues describe how dopamine-releasing neurons, which are understood to produce critical teaching signals for the brain, weigh the ambiguity of sensory information when they assess how successfully past experiences have guided a new decision. Their findings indicate that these neurons are even more sophisticated than previously thought.

"Dopamine neurons are not just any neurons in the brain," Kepecs says. "They are neurons that sit in the midbrain and send connections to large swaths of the brain. Dopamine neurons compare the predicted outcomes to actual outcomes, and send the discrepancy between these as an error feedback to other parts of the brain. This is exactly what you would want learning signals to do. And we've found that what dopamine neurons do is in perfect agreement with what is theoretically required for a powerful learning algorithm."

This is such an effective way of learning that the same strategy, called reinforcement learning, has been incorporated into many types of artificial intelligence, allowing computers to figure out how to do things without instructions from a programmer.

Kepecs and his team added what computer scientists call a belief state (a probability distribution variable) to a mathematical model of reinforcement learning. The previously used model is good at explaining how an animal learns from experience - but only when the sensory stimuli available to the animal are straightforward, easy to interpret. "It's been unclear whether or how dopamine neurons factor perceptual ambiguity into their teaching signal," Kepecs says. "Our revised model generates an estimate of the probability that a given choice is correct - this is the degree of confidence about the decision."

"If you can evaluate your decision process--the noise and everything else that's contributed to it--and in addition to making a binary choice assign a graded level of confidence to the decision, it's essentially a prediction about your accuracy," he says. "It's a completely different computation than just using the cues to recall past experiences to make predictions."

Armin Lak, a British Iranian postdoctoral researcher working with Kepecs, developed a computer model that predicted how neurons would behave if they were performing this calculation. Then they compared those predictions to the activity of dopamine neurons in the brains of monkeys while they performed a task in which they made a decision informed by visual cues that were sometimes clear, and sometimes not.

The activity of the neurons was recorded by Kepecs' collaborators Kensaku Nomoto and Masamichi Sakagami, neuroscientists at Tamagawa University in Tokyo, who were studying how the brain uses sensory information to make decisions. In the task, monkeys watched a screen filled with moving dots, and had to decide whether more dots were moving to the left or to the right, receiving a juice reward when they got the answer right.

When Kepecs and Lak analyzed the data, they found that the activity of the dopamine neurons closely fit the predictions of their new model--indicating that these neurons were likely evaluating the reliability of the information used during each decision. Applying the computational model was like looking at the data through a new microscope, Kepecs says. "By looking at it this way, you can see things that you couldn't see before in the data. Suddenly it just made sense. This system is much smarter than we thought before."

Surprisingly, Kepecs says, the confidence-signaling neurons became active a fraction of a second before the monkeys' made a decision. This early activity suggests the neurons may influence the immediate decision making process--a role that the team will explore in future research.
-end-
The research described here was supported by the Sir Henry Wellcome Trust (Postdoctoral Fellowship 106101/Z/14/Z); a Grant-in-Aid for the Center for Excellence Research from the Ministry of Education, Culture, Sports, Science, and Technology of Japan; the Swartz Foundation; and the National Institute of Mental Health (NIH grant R01MH097061).

"Midbrain Dopamine Neurons Signal Belief in Choice Accuracy during a Perceptual Decision" appears March 9, 2017 in Current Biology. The authors are Armin Lak, Kensaku Nomoto, Mehdi Keramati, Masamichi Sakagami, and Adam Kepecs. The paper can be accessed at http://www.cell.com/current-biology/home

About Cold Spring Harbor Laboratory

Founded in 1890, Cold Spring Harbor Laboratory has shaped contemporary biomedical research and education with programs in cancer, neuroscience, plant biology and quantitative biology. Home to eight Nobel Prize winners, the private, not-for-profit Laboratory employs 1,100 people including 600 scientists, students and technicians. The Meetings & Courses Program hosts more than 12,000 scientists from around the world each year on its campuses in Long Island and in Suzhou, China. The Laboratory's education arm also includes an academic publishing house, a graduate school and programs for middle and high school students and teachers. For more information, visit http://www.cshl.edu

Cold Spring Harbor Laboratory

Related Neurons Articles:

Shaping the social networks of neurons
Identification of a protein complex that attracts or repels nerve cells during development.
With these neurons, extinguishing fear is its own reward
The same neurons responsible for encoding reward also form new memories to suppress fearful ones, according to new research by scientists at The Picower Institute for Learning and Memory at MIT.
How do we get so many different types of neurons in our brain?
SMU (Southern Methodist University) researchers have discovered another layer of complexity in gene expression, which could help explain how we're able to have so many billions of neurons in our brain.
These neurons affect how much you do, or don't, want to eat
University of Arizona researchers have identified a network of neurons that coordinate with other brain regions to influence eating behaviors.
Mood neurons mature during adolescence
Researchers have discovered a mysterious group of neurons in the amygdala -- a key center for emotional processing in the brain -- that stay in an immature, prenatal developmental state throughout childhood.
Astrocytes protect neurons from toxic buildup
Neurons off-load toxic by-products to astrocytes, which process and recycle them.
Connecting neurons in the brain
Leuven researchers uncover new mechanisms of brain development that determine when, where and how strongly distinct brain cells interconnect.
The salt-craving neurons
Pass the potato chips, please! New research discovers neural circuits that regulate craving and satiation for salty tastes.
When neurons are out of shape, antidepressants may not work
Selective serotonin reuptake inhibitors (SSRIs) are the most commonly prescribed medication for major depressive disorder (MDD), yet scientists still do not understand why the treatment does not work in nearly thirty percent of patients with MDD.
Losing neurons can sometimes not be that bad
Current thinking about Alzheimer's disease is that neuronal cell death in the brain is to blame for the cognitive havoc caused by the disease.
More Neurons News and Neurons Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Uncharted
There's so much we've yet to explore–from outer space to the deep ocean to our own brains. This hour, Manoush goes on a journey through those uncharted places, led by TED Science Curator David Biello.
Now Playing: Science for the People

#556 The Power of Friendship
It's 2020 and times are tough. Maybe some of us are learning about social distancing the hard way. Maybe we just are all a little anxious. No matter what, we could probably use a friend. But what is a friend, exactly? And why do we need them so much? This week host Bethany Brookshire speaks with Lydia Denworth, author of the new book "Friendship: The Evolution, Biology, and Extraordinary Power of Life's Fundamental Bond". This episode is hosted by Bethany Brookshire, science writer from Science News.
Now Playing: Radiolab

Dispatch 1: Numbers
In a recent Radiolab group huddle, with coronavirus unraveling around us, the team found themselves grappling with all the numbers connected to COVID-19. Our new found 6 foot bubbles of personal space. Three percent mortality rate (or 1, or 2, or 4). 7,000 cases (now, much much more). So in the wake of that meeting, we reflect on the onslaught of numbers - what they reveal, and what they hide.  Support Radiolab today at Radiolab.org/donate.