Nav: Home

Evaluation of scientific rigor in animal research

December 02, 2016

The "reproducibility crisis" in biomedical research has led to questions about the scientific rigor in animal research, and thus the ethical justification of animal experiments. In research publishing in the Open Access journals PLOS Biology and PLOS ONE on December 2nd, 2016, researchers from the University of Bern have assessed scientific rigor in animal experimentation in Switzerland. The study, commissioned by the Swiss Federal Food Safety and Veterinary Office (FSVO), found widespread deficiencies in the reporting of experimental methodology.

In a first step, PhD student Lucile Vogt and postdoc Thomas Reichlin from the Division of Animal Welfare at the Vetsuisse Faculty in Bern screened all 1,277 approved applications for animal experiments in Switzerland in 2008, 2010 and 2012, as well as a random sample of 50 scientific publications resulting from studies described in the applications. The materials were assessed to determine whether seven basic methods that can help combat experimental bias were reported (including randomization, blinding, and sample size calculation). Appropriate use and understanding of these methods is a prerequisite for unbiased, scientifically valid results, says lead author Prof. Hanno Würbel, director of the Division of Animal Welfare.

As published in their PLOS Biology study, explicit evidence that these methods were used either in the applications for animal experiments or in the subsequent publications was scarce. For example, fewer than 20% of applications and publications mentioned whether a sample size calculation had been performed (8% in applications, 0% in publications), whether the animals had been assigned randomly to treatment groups (13% in applications, 17% in publications), and whether outcome assessment had been conducted blind to treatment (3% in applications, 11% in publications).

Animal experiments are authorized based on the explicit understanding that they will provide significant new knowledge and that animals will suffer no unnecessary harm. Thus, scientific rigor is a fundamental prerequisite for the ethical justification of animal experiments. Based on this study, the current practice of authorizing animal experiments appears to rest on an assumption of scientific rigor, rather than on evidence that it is applied. The authors of this study recommend more education and training in good research practice and scientific integrity for all those involved in this process.

Although the initial results found that fewer than 20 percent of applications and publications used methods to control for bias, that didn't necessarily mean that more than 80 percent of animal studies failed to include methods to combat bias, and therefore use animals for potentially inconclusive research. "It is possible that the researchers did use these methods but did not mention them in their applications and publications," says study director Hanno Würbel. "So we decided to ask the researchers."

The researchers used an online survey for all 1,891 animal researchers registered in the central online information system of the FSVO who were involved with ongoing experiments. Among other questions, researchers were asked what bias-reducing methods they normally use when conducting animal experiments and which of these they had explicitly reported in their latest scientific publication.

According to the researchers' responses, as published in their PLOS ONE study, the use of methods against bias is considerably higher than reported in the animal research applications and publications. 86% of the participants claimed to assign animals randomly to treatment groups, but only 44% answered that they had reported this in their latest publication. The same applies to the other measures, for example, for sample size calculation (69% claimed to be doing this, but only 18% say they reported it in their latest publication) and for blinded outcome assessment (47% vs. 27%).

Taken together, the researchers draw two conclusions from these results: on the one hand, reporting in animal research applications or publications may underestimate the use of bias-reducing methods. On the other hand, the researchers may overestimate their use of appropriate methods. "We found considerably fewer publications with explicit evidence of the use of measures against risks of bias than claimed by the researchers", says Würbel. For example, 44% of the participants claimed to have reported randomization in their latest publication, but Würbel's team found evidence of randomization in only 17% of publications.
In your coverage please use this URL to provide access to the freely available article in PLOS Biology: and this URL to provide access to the freely available article in PLOS ONE:

Citations: Vogt L, Reichlin TS, Nathues C, Würbel H (2016) Authorization of Animal Experiments Is Based on Confidence Rather than Evidence of Scientific Rigor. PLoS Biol 14(12): e2000598. doi:10.1371/journal.pbio.2000598

Reichlin TS, Vogt L, Würbel H (2016) The Researchers' View of Scientific Rigor--Survey on the Conduct and Reporting of In Vivo Research. PLoS ONE 11(12): e0165999. doi:10.1371/journal.pone.0165999

Funding: Swiss Food Safety and Veterinary Office (FSVO) (grant number 2.13.01). LV and TSR were fully funded through this grant. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing Interests: The authors have declared that no competing interests exist.


Related Bias Articles:

Bullying and bias can cost schools millions in lost funding
When children avoid school to avoid bullying, many states can lose tens of millions of dollars in lost funding, and California alone loses an estimated $276 million each year because children feel unsafe.
White people show race bias when judging deception
When making judgments about who is lying and who is telling the truth, new research shows that White people are more likely to label a Black person as a truth-teller compared with a White person, even though their spontaneous behavior indicates the reverse bias.
No evidence of gender bias in philosophy
Despite being a male-dominated field, a recent study found no evidence for gender bias against women in philosophy, in terms of securing tenure-track positions as college professors.
Study finds gender bias in open-source programming
A study comparing acceptance rates of contributions from men and women in an open-source software community finds that, overall, women's contributions tend to be accepted more often than men's -- but when a woman's gender is identifiable, they are rejected more often.
Infants show racial bias toward members of own race and against those of other races
Two studies by researchers at the Ontario Institute for Studies in Education (OISE) at the University of Toronto and their collaborators from the US, UK, France and China, show that six- to nine-month-old infants demonstrate racial bias in favor of members of their own race and racial bias against those of other races.
No publication bias found in climate change research
Rarely do we encounter a scientific fact that stirs public controversy and distrust in science as much as climate change.
Evaluation of emergency medicine residents points to gender bias
By the end of the third and final year of residency, evaluations of female physicians placed them three to four months behind male colleagues in the same training program.
Analysis uncovers racial bias in fatal shootings by police
A recent analysis found that among 990 individuals fatally shot by US police officers in 2015, Black civilians were more than twice as likely as White civilians to have been unarmed, and civilians from 'other' minority groups were significantly more likely than White civilians to have not posed an imminent threat to the officer(s) or other civilians.
Does 'publication bias' affect the 'canonization' of facts in science?
In an article published Dec. 20 in the journal eLife, UW biology professor Carl Bergstrom and co-authors present a mathematical model that explores whether 'publication bias' -- the tendency of journals to publish mostly positive experimental results -- influences how scientists canonize facts.
Are death row cases plagued with racial bias?
Defendants charged with murder in North Carolina from 1990 to 2009 were more than twice as likely to receive the death penalty if the victims were white, Michigan State University researchers have found.

Related Bias Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Jumpstarting Creativity
Our greatest breakthroughs and triumphs have one thing in common: creativity. But how do you ignite it? And how do you rekindle it? This hour, TED speakers explore ideas on jumpstarting creativity. Guests include economist Tim Harford, producer Helen Marriage, artificial intelligence researcher Steve Engels, and behavioral scientist Marily Oppezzo.
Now Playing: Science for the People

#524 The Human Network
What does a network of humans look like and how does it work? How does information spread? How do decisions and opinions spread? What gets distorted as it moves through the network and why? This week we dig into the ins and outs of human networks with Matthew Jackson, Professor of Economics at Stanford University and author of the book "The Human Network: How Your Social Position Determines Your Power, Beliefs, and Behaviours".