Nav: Home

Study: Conflict of interest disclosures don't alter the recommendations of peer reviewers

November 07, 2019

The majority of high-quality medical and science journals require disclosure of possible conflicts of interest (COI). However, a new study suggests that such disclosures have no impact on journal reviewers, even when the authors of submitted papers did, in fact, report conflicts. The study also found that reviewers' evaluations of seven additional measures of different facets of research quality (e.g., methods, conclusions, objectivity) were similarly unaffected by COI disclosures.

The study was conducted by researchers at Carnegie Mellon University, Harvard and the University of California, San Francisco (UCSF). It examined the evaluations of more than 3,300 reviewers of nearly 1,500 papers submitted between 2014 and 2018 to the Annals of Emergency Medicine (AEM). The authors believe it is the first randomly controlled experiment to examine the impact of COI disclosures on actual reviewer evaluations of research papers.

In the study, reviewers received a manuscript and were either informed within 24 hours if the authors had or had not revealed a COI during the submission process or were not informed of the authors' COIs one way or the other. Nearly one third of the papers had disclosures, primarily regarding funding sources. After returning their evaluation of the research, all reviewers were surveyed.

Although 78 percent of the reviewers in the first group indicated they had read and understood the COI disclosures, and most also agreed that COIs are an important issue, the disclosures had no effect on reviewers' recommendations regarding papers' suitability for publication.

The study was led by Harvard's Leslie John, who earned her Ph.D. in Behavioral Decision Research at Carnegie Mellon in 2011. She said the findings likely are related to the ambiguity of the review process as it relates to COI disclosures.

"Reviewers aren't given any explicit guidance on how to correct for possible COI," said John, the Marvin Bower Associate Professor at Harvard Business School. "If a disclosure reveals a significant COI, should the reviewer recommend rejection or apply special scrutiny to methods? There currently isn't a consensus about how reviewers should respond to COI disclosures, nor any direction provided about how to do so."

John added that most of the reviewers reported that they strongly believed they could correct for the biasing influence of conflicts of interest if and when they were disclosed.

"Even experienced peer reviewers usually cannot articulate a specific and effective approach to follow once COI is suspected or declared," said Michael Callaham, AEM's editor-in-chief, emeritus chair of emergency medicine at UCSF and a senior author of the paper. "As a result, most reviewers do not change their assessments due to this information. Clearly the guidelines for reporting COI are not achieving their goal and need to be further studied and improved."

George Loewenstein, Carnegie Mellon's Herbert A. Simon University Professor of Economics and Psychology and a co-author of the study, noted that the paper is the latest in a long line of research showing that disclosure is an inadequate response to COIs that are rampant in medicine.

"The medical and academic establishments think they've dealt with the issues of COIs by including disclosures," Loewenstein commented. "This experiment provides further evidence that they haven't."
The study's fourth author, Andrew Marder, worked on the experiment as a senior statistician at Harvard Business School. The paper, "The Impact of Revealing Authors' Conflicts of Interests in Peer Review: A Randomised Controlled Trial," now appears online in the British Medical Journal.

Carnegie Mellon University

Related Emergency Medicine Articles:

The cost of waiting in emergency departments
Wait times in US emergency departments are increasing. A new study published in Economic Inquiry indicates that prolonging the wait time in the emergency department for a patient who arrives with a serious condition by 10 minutes will increase the hospital's cost to care for the patient by an average of 6%, and it will increase the cost to care for moderately severe cases by an average of 3%.
Emergency medicine: Department-based intensive care unit improves patient survival rates
A new Michigan Medicine study found that implementing a dedicated emergency medicine department-based intensive care unit improved patient survival rates and lowered inpatient intensive care unit (ICU) admissions.
Emergency room or doctor's office?
A new study in the journal Heliyon, published by Elsevier, examines the relationship between the way individuals perceive and respond to threats (threat sensitivity) and where they most frequently seek medical care.
Gender-based salary gap persists among academic emergency medicine physicians
Although overall salaries for emergency physicians have increased over the past four years, and despite a call to end gender disparities in salary, men still make 18 percent more than women, and a $12,000 gender salary gap remains essentially unchanged.
Study reveals complementary medicine use remains hidden to conventional medicine providers
Research reveals that 1 in 3 complementary medicine (CM) users do not disclose their CM use to their medical providers, posing significant direct and indirect risks of adverse effects and harm due to unsafe concurrent use of CM and conventional medicine use.
Study of traditional medicine finds high use in Sub-Saharan Africa despite modern medicine
Researchers who have undertaken the first systematic review of into the use of traditional, complementary and alternative medicines (TCAM) in Sub-Saharan Africa found its use is significant and not just because of a lack of resources or access to 'conventional medicine'.
CU School of Medicine's Kenneth Tyler article in New England Journal of Medicine
Kenneth Tyler, M.D., the Louise Baum Endowed Chair in Neurology at the University of Colorado School of Medicine, is author of a review article about acute viral encephalitis in the current issue of The New England Journal of Medicine.
New program keeps elderly out of emergency
A medical program developed by emergency and palliative care clinicians at a large Australian hospital is seeing elderly aged care residents successfully treated at home.
Society for Academic Emergency Medicine announces Annual Meeting plenary speakers
Emergency medicine academicians in six plenary presentations will explore a variety of subjects related to the practice of emergency medicine during a special plenary session to be held on the opening day of SAEM18--the annual meeting for the Society for Academic Emergency Medicine (SAEM) and the largest forum for the presentation of original education and research in academic emergency medicine.
One in four emergency staff abused by patients
The experience of hospital A&E staff reveals that they have resigned themselves to patient violence and aggression.
More Emergency Medicine News and Emergency Medicine Current Events

Top Science Podcasts

We have hand picked the top science podcasts of 2019.
Now Playing: TED Radio Hour

Why do we revere risk-takers, even when their actions terrify us? Why are some better at taking risks than others? This hour, TED speakers explore the alluring, dangerous, and calculated sides of risk. Guests include professional rock climber Alex Honnold, economist Mariana Mazzucato, psychology researcher Kashfia Rahman, structural engineer and bridge designer Ian Firth, and risk intelligence expert Dylan Evans.
Now Playing: Science for the People

#541 Wayfinding
These days when we want to know where we are or how to get where we want to go, most of us will pull out a smart phone with a built-in GPS and map app. Some of us old timers might still use an old school paper map from time to time. But we didn't always used to lean so heavily on maps and technology, and in some remote places of the world some people still navigate and wayfind their way without the aid of these tools... and in some cases do better without them. This week, host Rachelle Saunders...
Now Playing: Radiolab

Dolly Parton's America: Neon Moss
Today on Radiolab, we're bringing you the fourth episode of Jad's special series, Dolly Parton's America. In this episode, Jad goes back up the mountain to visit Dolly's actual Tennessee mountain home, where she tells stories about her first trips out of the holler. Back on the mountaintop, standing under the rain by the Little Pigeon River, the trip triggers memories of Jad's first visit to his father's childhood home, and opens the gateway to dizzying stories of music and migration. Support Radiolab today at