Nav: Home

Studies of brain activity aren't as useful as scientists thought

June 03, 2020

DURHAM, N.C. - Hundreds of published studies over the last decade have claimed it's possible to predict an individual's patterns of thoughts and feelings by scanning their brain in an MRI machine as they perform some mental tasks.

But a new analysis by some of the researchers who have done the most work in this area finds that those measurements are highly suspect when it comes to drawing conclusions about any individual person's brain.

Watching the brain through a functional MRI machine (fMRI) is still great for finding the general brain structures involved in a given task across a group of people, said Ahmad Hariri, a professor of psychology and neuroscience at Duke University who led the reanalysis.

"Scanning 50 people is going to accurately reveal what parts of the brain, on average, are more active during a mental task, like counting or remembering names," Hariri said

Functional MRI measures blood flow as a proxy for brain activity. It shows where blood is being sent in the brain, presumably because neurons in that area are more active during a mental task.

The problem is that the level of activity for any given person probably won't be the same twice, and a measure that changes every time it is collected cannot be applied to predict anyone's future mental health or behavior.

Hariri and his colleagues reexamined 56 published papers based on fMRI data to gauge their reliability across 90 experiments. Hariri said the researchers recognized that "the correlation between one scan and a second is not even fair, it's poor."

They also examined data from the brain-scanning Human Connectome Project -- "Our field's Bible at the moment," Hariri called it -- and looked at test/retest results for 45 individuals. For six out of seven measures of brain function, the correlation between tests taken about four months apart with the same person was weak. The seventh measure studied, language processing, was only a fair correlation, not good or excellent.

Finally they looked at data they collected through the Dunedin Multidisciplinary Health and Development Study in New Zealand, in which 20 individuals were put through task-based fMRI twice, two or three months apart. Again, they found poor correlation from one test to the next in an individual.

The bottom line is that task-based fMRI in its current form can't tell you what an individual's brain activation will look like from one test to the next, Hariri said. The new analysis, appears June 3 in Psychological Science

"This is more relevant to my work than just about anyone else's!" Hariri said, his voice rising. "This is my fault. I'm going to throw myself under the bus. This whole sub-branch of fMRI could go extinct if we can't address this critical limitation."

Hariri has been using fMRI data as part of a long-term study of 1,300 undergraduate Duke students. By combining brain scans, genetic testing and psychological assessments, Hariri is searching for biomarkers of individual differences in the way people process thoughts and emotions, such as why one person comes away from a traumatic event with PTSD or depression and another does not.

"We can't continue with the same old 'hot spot' research," Hariri said. "We could scan the same 1,300 undergrads again and we wouldn't see the same patterns for each of them."

One possible solution to the reliability problem, using existing technology, would be to collect data for a full hour or longer in the scanner, not just five minutes. Hariri also said developing new tasks from the ground up with the explicit purpose of reliably measuring individual differences in brain activity is another strategy. In the meanwhile, Hariri and his team have shifted their focus to MRI measures of brain structure, which are highly reliable.

"It's not as if we haven't known these issues of reliability, but this paper brings them together more sharply," said Russell Poldrack, the Albert Ray Lang Professor of Psychology at Stanford University, who had a 15-year-old fMRI paper among those that were reanalyzed.

"This is a good wakeup call, and it's a marker of Ahmad's integrity that he's taking this on," said Poldrack, who was not involved in the meta-analysis but said he has had suspicions about fMRI reliability for a few years now.

Connectivity mapping - seeing how areas of the brain are connected to address a task more than just what areas are active - is going to be the way forward, Poldrack predicted. Hariri agreed that identifying patterns of activity throughout the brain rather than in one or two areas may improve reliability.

In the meantime, the sociology behind a dramatic debunking of a scientific tool is going to be interesting to watch, Hariri and Poldrack both said.

"There's three things you can do," Poldrack said. "You can just up and quit, you can stick your head in the sand (and act as if nothing has changed), or you can dig in and try to solve the problems."
-end-
This analysis was supported by the U.S. National Science Foundation. The Dunedin Study is supported by the U.S. National Institute on Aging (R01AG049789, R01AG032282) and the UK Medical Research Council (P005918), the New Zealand Health Research Council and the New Zealand Ministry of Business, Innovation and Employment (MBIE). The Human Connectome Project is supported by 16 centers of the U.S. National Institutes of Health via the Blueprint for Neuroscience Research.

CITATION: "What is the Test-Retest Reliability of Common Task-fMRI Measure? New Empirical Evidence and a Meta-Analysis," Maxwell L. Elliott, Annchen R. Knodt, David Ireland, Meriwether L. Morris, Richie Poulton, Sandhya Ramrakha, Maria L. Sison, Terrie E. Moffitt, Avshalom Caspi, Ahmad R. Hariri. Psychological Science, June 3, 2020. DOI: 10.1177/0956797620916786

Duke University

Related Brain Activity Articles:

What is your attitude towards a humanoid robot? Your brain activity can tell us!
Researchers at IIT-Istituto Italiano di Tecnologia in Italy found that people's bias towards robots, that is, attributing them intentionality or considering them as 'mindless things', can be correlated with distinct brain activity patterns.
Using personal frequency to control brain activity
Individual frequency can be used to specifically influence certain areas of the brain and thus the abilities processed in them - solely by electrical stimulation on the scalp, without any surgical intervention.
Rats' brain activity reveals their alcohol preference
The brain's response to alcohol varies based on individual preferences, according to new research in rats published in eNeuro.
Studies of brain activity aren't as useful as scientists thought
Hundreds of published studies over the last decade have claimed it's possible to predict an individual's patterns of thoughts and feelings by scanning their brain in an MRI machine as they perform some mental tasks.
A child's brain activity reveals their memory ability
A child's unique brain activity reveals how good their memories are, according to research recently published in JNeurosci.
How dopamine drives brain activity
Using a specialized magnetic resonance imaging (MRI) sensor that can track dopamine levels, MIT neuroscientists have discovered how dopamine released deep within the brain influences distant brain regions.
Brain activity intensity drives need for sleep
The intensity of brain activity during the day, notwithstanding how long we've been awake, appears to increase our need for sleep, according to a new UCL study in zebrafish, published in Neuron.
Do babies like yawning? Evidence from brain activity
Contagious yawning is observed in many mammals, but there is no such report in human babies.
Understanding brain activity when you name what you see
Using complex statistical methods and fast measurement techniques, researchers found how the brain network comes up with the right word and enables us to say it.
Your brain activity can be used to measure how well you understand a concept
As students learn a new concept, measuring how well they grasp it has often depended on traditional paper and pencil tests.
More Brain Activity News and Brain Activity Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Power Of Spaces
How do spaces shape the human experience? In what ways do our rooms, homes, and buildings give us meaning and purpose? This hour, TED speakers explore the power of the spaces we make and inhabit. Guests include architect Michael Murphy, musician David Byrne, artist Es Devlin, and architect Siamak Hariri.
Now Playing: Science for the People

#576 Science Communication in Creative Places
When you think of science communication, you might think of TED talks or museum talks or video talks, or... people giving lectures. It's a lot of people talking. But there's more to sci comm than that. This week host Bethany Brookshire talks to three people who have looked at science communication in places you might not expect it. We'll speak with Mauna Dasari, a graduate student at Notre Dame, about making mammals into a March Madness match. We'll talk with Sarah Garner, director of the Pathologists Assistant Program at Tulane University School of Medicine, who takes pathology instruction out of...
Now Playing: Radiolab

What If?
There's plenty of speculation about what Donald Trump might do in the wake of the election. Would he dispute the results if he loses? Would he simply refuse to leave office, or even try to use the military to maintain control? Last summer, Rosa Brooks got together a team of experts and political operatives from both sides of the aisle to ask a slightly different question. Rather than arguing about whether he'd do those things, they dug into what exactly would happen if he did. Part war game part choose your own adventure, Rosa's Transition Integrity Project doesn't give us any predictions, and it isn't a referendum on Trump. Instead, it's a deeply illuminating stress test on our laws, our institutions, and on the commitment to democracy written into the constitution. This episode was reported by Bethel Habte, with help from Tracie Hunte, and produced by Bethel Habte. Jeremy Bloom provided original music. Support Radiolab by becoming a member today at Radiolab.org/donate.     You can read The Transition Integrity Project's report here.