Scientists don't turn a blind eye to bias

July 09, 2015

Scientific journals should insist on more robust experimental processes, say biologists after reviewing nearly 900,000 experiments.

The team found that non-blind experiments - that is, where scientists knew which samples they were recording - averaged a 27 per cent stronger result than blind trials.

However their review suggests that less than one in four experiments used blind data recording.

"We found that non-blind papers tended to exaggerate differences between the experimental group and the control group," said lead researcher Dr Luke Holman, from the Research School of Biology at The Australian National University (ANU).

"For example, a non-blind trial of a new drug might conclude that it is way more effective than a placebo, when in fact the drug's true effect is rather modest, simply because the researchers' expectations biased the results."

The paper is published in PLOS Biology at a time when experimental processes are under the microscope following increased levels of retractions and some journals are reviewing their peer review procedures.

In the largest study of its kind, the team analysed nearly 900,000 papers from the PubMed life sciences database, using automated 'data mining'. They also - in a blind trial, of course - compared 83 pairs of evolutionary biology papers on similar topics, in which the data was collected blind in one, and not in the other.

The team also found that non-blind studies rejected the null hypothesis more strongly, said Dr Holman.

"Non-blind studies more confidently concluded that differences between treatment and control groups were real, and not just due to chance variation."

Co-researcher Dr Megan Head, also from ANU Research School of Biology said self-reflection is important.

"Science is still the best method we have for understanding the world, and we have to keep working to make it better."

Dr Holman and his colleagues believe that journals should insist on blind trials more strongly, perhaps by making prominent statements to authors and peer reviewers about the necessity of using blind trials.

Scientists are aware of their biases, and use techniques such as blind trials to minimise them, but the pressure to get things done faster leads to some people skimping on experimental design, said Dr Head.

"It is not necessarily slower to take data blind, you just need to be a little creative," she said.

Dr Holman suggested better training is part of the solution.

"Many researchers are unaware that their expectations can introduce such strong bias, and so they don't feel the need to work blind," he said.
-end-


Australian National University

Related Blind Articles from Brightsurf:

Identifying the blind spots of soil biodiversity
Soils harbour a substantial part of the world's biodiversity, yet data on the patterns and processes taking place below ground does not represent all relevant ecosystems and taxa.

Strong convictions can blind us to information that challenges them
When people are highly confident in a decision, they take in information that confirms their decision, but fail to process information which contradicts it, finds a UCL brain imaging study, published in Nature Communications.

A tactile robot finger with no blind spots
Researchers at Columbia Engineering announced today that they have introduced a new type of robotic finger with a sense of touch.

Exposing blind spots in the carbon budget space
The impact of 1°C of global heating is already having devastating impacts on communities and ecosystems across the globe.

Optic nerve stimulation to aid the blind
EPFL scientists are investigating new ways to provide visual signals to the blind by directly stimulating the optic nerve.

Identifying artificial intelligence 'blind spots'
A novel model developed by MIT and Microsoft researchers identifies instances in which autonomous systems have 'learned' from training examples that don't match what's actually happening in the real world.

Seeing and avoiding the 'blind spot' in atomic force measurements
Researchers have discovered a 'blind spot' in atomic force microscopy -- a powerful tool capable of measuring the force between two atoms, imaging the structure of individual cells and the motion of biomolecules.

Giant flightless birds were nocturnal and possibly blind
If you encountered an elephant bird today, it would be hard to miss.

'Blind' Cheetah 3 robot can climb stairs littered with obstacles
MIT's Cheetah 3 robot can now leap and gallop across rough terrain, climb a staircase littered with debris, and quickly recover its balance when suddenly yanked or shoved, all while essentially blind.

Researchers map brain of blind patient who can see motion
Since the visual processing centres of her brain went dark after a stroke, a Scottish woman has been unable to see objects.

Read More: Blind News and Blind Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.