Nav: Home

University of Idaho study finds scientific reproducibility does not equate to scientific truth

May 15, 2019

MOSCOW, Idaho -- May 15, 2019 -- Reproducible scientific results are not always true and true scientific results are not always reproducible, according to a mathematical model produced by University of Idaho researchers. Their study, which simulates the search for that scientific truth, will be published Wednesday, May 15, in the journal PLOS ONE. It is embargoed until 2 p.m. ET that day.

Independent confirmation of scientific results -- known as reproducibility -- lends credibility to a researcher's conclusion. But researchers have found the results of many well-known science experiments cannot be reproduced, an issue referred to as a "replication crisis."

"Over the last decade, people have focused on trying to find remedies for the 'replication crisis,'" said Berna Devezer, lead author of the study and U of I associate professor of marketing in the College of Business and Economics. "But proposals for remedies are being accepted and implemented too fast without solid justifications to support them. We need a better theoretical understanding of how science operates before we can provide reliable remedies for the right problems. Our model is a framework for studying science."

Devezer and her colleagues investigated the relationship between reproducibility and the discovery of scientific truths by building a mathematical model that represents a scientific community working toward finding a scientific truth. In each simulation, the scientists are asked to identify the shape of a specific polygon.

The modeled scientific community included multiple scientist types, each with a different research strategy, such as performing highly innovative experiments or simple replication experiments. Devezer and her colleagues studied whether factors like the makeup of the community, the complexity of the polygon and the rate of reproducibility influenced how fast the community settled on the true polygon shape as the scientific consensus and the persistence of the true polygon shape as the scientific consensus.

Within the model, the rate of reproducibility did not always correlate with the probability of identifying the truth, how fast the community identified the truth and whether the community stuck with the truth once they identified it. These findings indicate reproducible results are not synonymous with finding the truth, Devezer said.

Compared to other research strategies, highly innovative research tactics resulted in a quicker discovery of the truth. According to the study, a diversity of research strategies protected against ineffective research approaches and optimized desirable aspects of the scientific process.

Variables including the makeup of the community and complexity of the true polygon influenced the speed scientists discovered the truth and persistence of that truth, suggesting the validity of scientific results should not be automatically blamed on questionable research practices or problematic incentives, Devezer said. Both have been pointed to as drivers of the "replication crisis."

"We found that, within the model, some research strategies that lead to reproducible results could actually slow down the scientific process, meaning reproducibility may not always be the best -- or at least the only -- indicator of good science," said Erkan Buzbas, U of I assistant professor in the College of Science, Department of Statistical Science and a co-author on the paper. "Insisting on reproducibility as the only criterion might have undesirable consequences for scientific progress."
-end-
Link to the article, which will go live upon publication: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0216125

This project was funded under National Institutes of Health, National Institute of General Medical Sciences grant No. P20GM104420. The total amount of federal funding for the overall project is $10,572,579, of which 100 percent is the federal share.

About the University of Idaho

The University of Idaho, home of the Vandals, is Idaho's land-grant, national research university. From its residential campus in Moscow, U of I serves the state of Idaho through educational centers in Boise, Coeur d'Alene and Idaho Falls, nine research and Extension centers, plus Extension offices in 42 counties. Home to nearly 12,000 students statewide, U of I is a leader in student-centered learning and excels at interdisciplinary research, service to businesses and communities, and in advancing diversity, citizenship and global outreach. U of I competes in the Big Sky Conference. Learn more at http://www.uidaho.edu.

University of Idaho

Related Mathematical Model Articles:

How big brains evolved could be revealed by new mathematical model
A new mathematical model could help clarify what drove the evolution of large brains in humans and other animals, according to a study published in PLOS Computational Biology.
'Field patterns' as a new mathematical object
University of Utah mathematicians propose a theoretical framework to understand how waves and other disturbances move through materials in conditions that vary in both space and time.
Mathematical model limits malaria outbreaks
Mathematical models can effectively predict and track malaria transmission trends, ultimately quantifying the efficiency of various treatment and eradication strategies in high-risk regions.
Scientists create first viable mathematical model of a key anti-Salmonella defense system
Scientists have created the first validated mathematical model of an important cellular defense mechanism against the bacterium Salmonella, according to a new study in PLOS Computational Biology.
Mathematical algorithms calculate social behavior
For a long time, mathematical modelling of social systems and dynamics was considered in the realm of science fiction.
Researchers create first 3-D mathematical model of uterine contractions
By studying the electric activity that causes uterine contractions in pregnant women, researchers at Washington University in St.
Exploring the mathematical universe
A team of more than 80 mathematicians from 12 countries has begun charting the terrain of rich, new mathematical worlds, and sharing their discoveries on the Web.
New mathematical model challenges aggressive antibiotic treatments
Antibiotic resistance is one of the most challenging problems in modern medicine.
A mathematical advance in describing waves
Two UB mathematicians have published a new paper that advances the art -- or shall we say, the math -- of describing a wave.

Related Mathematical Model Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Digital Manipulation
Technology has reshaped our lives in amazing ways. But at what cost? This hour, TED speakers reveal how what we see, read, believe — even how we vote — can be manipulated by the technology we use. Guests include journalist Carole Cadwalladr, consumer advocate Finn Myrstad, writer and marketing professor Scott Galloway, behavioral designer Nir Eyal, and computer graphics researcher Doug Roble.
Now Playing: Science for the People

#529 Do You Really Want to Find Out Who's Your Daddy?
At least some of you by now have probably spit into a tube and mailed it off to find out who your closest relatives are, where you might be from, and what terrible diseases might await you. But what exactly did you find out? And what did you give away? In this live panel at Awesome Con we bring in science writer Tina Saey to talk about all her DNA testing, and bioethicist Debra Mathews, to determine whether Tina should have done it at all. Related links: What FamilyTreeDNA sharing genetic data with police means for you Crime solvers embraced...