Nav: Home

University of Idaho study finds scientific reproducibility does not equate to scientific truth

May 15, 2019

MOSCOW, Idaho -- May 15, 2019 -- Reproducible scientific results are not always true and true scientific results are not always reproducible, according to a mathematical model produced by University of Idaho researchers. Their study, which simulates the search for that scientific truth, will be published Wednesday, May 15, in the journal PLOS ONE. It is embargoed until 2 p.m. ET that day.

Independent confirmation of scientific results -- known as reproducibility -- lends credibility to a researcher's conclusion. But researchers have found the results of many well-known science experiments cannot be reproduced, an issue referred to as a "replication crisis."

"Over the last decade, people have focused on trying to find remedies for the 'replication crisis,'" said Berna Devezer, lead author of the study and U of I associate professor of marketing in the College of Business and Economics. "But proposals for remedies are being accepted and implemented too fast without solid justifications to support them. We need a better theoretical understanding of how science operates before we can provide reliable remedies for the right problems. Our model is a framework for studying science."

Devezer and her colleagues investigated the relationship between reproducibility and the discovery of scientific truths by building a mathematical model that represents a scientific community working toward finding a scientific truth. In each simulation, the scientists are asked to identify the shape of a specific polygon.

The modeled scientific community included multiple scientist types, each with a different research strategy, such as performing highly innovative experiments or simple replication experiments. Devezer and her colleagues studied whether factors like the makeup of the community, the complexity of the polygon and the rate of reproducibility influenced how fast the community settled on the true polygon shape as the scientific consensus and the persistence of the true polygon shape as the scientific consensus.

Within the model, the rate of reproducibility did not always correlate with the probability of identifying the truth, how fast the community identified the truth and whether the community stuck with the truth once they identified it. These findings indicate reproducible results are not synonymous with finding the truth, Devezer said.

Compared to other research strategies, highly innovative research tactics resulted in a quicker discovery of the truth. According to the study, a diversity of research strategies protected against ineffective research approaches and optimized desirable aspects of the scientific process.

Variables including the makeup of the community and complexity of the true polygon influenced the speed scientists discovered the truth and persistence of that truth, suggesting the validity of scientific results should not be automatically blamed on questionable research practices or problematic incentives, Devezer said. Both have been pointed to as drivers of the "replication crisis."

"We found that, within the model, some research strategies that lead to reproducible results could actually slow down the scientific process, meaning reproducibility may not always be the best -- or at least the only -- indicator of good science," said Erkan Buzbas, U of I assistant professor in the College of Science, Department of Statistical Science and a co-author on the paper. "Insisting on reproducibility as the only criterion might have undesirable consequences for scientific progress."
Link to the article, which will go live upon publication:

This project was funded under National Institutes of Health, National Institute of General Medical Sciences grant No. P20GM104420. The total amount of federal funding for the overall project is $10,572,579, of which 100 percent is the federal share.

About the University of Idaho

The University of Idaho, home of the Vandals, is Idaho's land-grant, national research university. From its residential campus in Moscow, U of I serves the state of Idaho through educational centers in Boise, Coeur d'Alene and Idaho Falls, nine research and Extension centers, plus Extension offices in 42 counties. Home to nearly 12,000 students statewide, U of I is a leader in student-centered learning and excels at interdisciplinary research, service to businesses and communities, and in advancing diversity, citizenship and global outreach. U of I competes in the Big Sky Conference. Learn more at

University of Idaho

Related Mathematical Model Articles:

Moffitt mathematical model predicts patient outcomes to adaptive therapy
In an article published in Nature Communications, Moffitt Cancer Center researchers provide a closer look at a mathematical model and data showing that individual patient alterations in the prostate-specific antigen (PSA) biomarker early in cancer treatment can predict outcomes to later treatment cycles of adaptive therapy.
New mathematical model can more effectively track epidemics
As COVID-19 spreads worldwide, leaders are relying on mathematical models to make public health and economic decisions.
Mathematical model could lead to better treatment for diabetes
MIT researchers have developed a mathematical model that can predict the behavior of glucose-responsive insulin in humans and in rodents.
New mathematical model reveals how major groups arise in evolution
Researchers at Uppsala University and the University of Leeds presents a new mathematical model of patterns of diversity in the fossil record, which offers a solution to Darwin's ''abominable mystery'' and strengthens our understanding of how modern groups originate.
Mathematical model reveals behavior of cellular enzymes
Mathematical modeling helps researchers to understand how enzymes in the body work to ensure normal functioning.
New mathematical model for amyloid formation
Scientists report on a mathematical model for the formation of amyloid fibrils.
New mathematical model shows how diversity speeds consensus
Scientific literature abounds with examples of ways in which member diversity can benefit a group -- whether spider colonies' ability to forage or an industrial company's financial performance.
Newly developed mathematical model could be used to predict cancer drug side effects
A research team at Kobe University Hospital have further illuminated the likelihood of cancer drug side effects that can occur due to genetic mutations in the drug-metabolizing enzyme.
A mathematical model reveals long-distance cell communication mechanism
An interdisciplinary collaborative team at KAIST has identified how a large community can communicate with each other almost simultaneously even with very short distance signaling.
Experimentally validated model for drug discovery gets a stamp of mathematical approval
Insilico Medicine, a biotechnology company developing an end-to-end drug discovery pipeline utilizing next-generation artificial intelligence, is proud to present its paper 'A Prior of a Googol Gaussians: a Tensor Ring Induced Prior for Generative Models' at the 33rd Conference on Neural Information Processing Systems (NeurIPS).
More Mathematical Model News and Mathematical Model Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Biology Of Sex
Original broadcast date: May 8, 2020. Many of us were taught biological sex is a question of female or male, XX or XY ... but it's far more complicated. This hour, TED speakers explore what determines our sex. Guests on the show include artist Emily Quinn, journalist Molly Webster, neuroscientist Lisa Mosconi, and structural biologist Karissa Sanbonmatsu.
Now Playing: Science for the People

#569 Facing Fear
What do you fear? I mean really fear? Well, ok, maybe right now that's tough. We're living in a new age and definition of fear. But what do we do about it? Eva Holland has faced her fears, including trauma and phobia. She lived to tell the tale and write a book: "Nerve: Adventures in the Science of Fear".
Now Playing: Radiolab

The Wubi Effect
When we think of China today, we think of a technological superpower. From Huweai and 5G to TikTok and viral social media, China is stride for stride with the United States in the world of computing. However, China's technological renaissance almost didn't happen. And for one very basic reason: The Chinese language, with its 70,000 plus characters, couldn't fit on a keyboard.  Today, we tell the story of Professor Wang Yongmin, a hard headed computer programmer who solved this puzzle and laid the foundation for the China we know today. This episode was reported and produced by Simon Adler with reporting assistance from Yang Yang. Special thanks to Martin Howard. You can view his renowned collection of typewriters at: Support Radiolab today at