Nav: Home

Study: Conflict of interest disclosures don't alter the recommendations of peer reviewers

November 07, 2019

The majority of high-quality medical and science journals require disclosure of possible conflicts of interest (COI). However, a new study suggests that such disclosures have no impact on journal reviewers, even when the authors of submitted papers did, in fact, report conflicts. The study also found that reviewers' evaluations of seven additional measures of different facets of research quality (e.g., methods, conclusions, objectivity) were similarly unaffected by COI disclosures.

The study was conducted by researchers at Carnegie Mellon University, Harvard and the University of California, San Francisco (UCSF). It examined the evaluations of more than 3,300 reviewers of nearly 1,500 papers submitted between 2014 and 2018 to the Annals of Emergency Medicine (AEM). The authors believe it is the first randomly controlled experiment to examine the impact of COI disclosures on actual reviewer evaluations of research papers.

In the study, reviewers received a manuscript and were either informed within 24 hours if the authors had or had not revealed a COI during the submission process or were not informed of the authors' COIs one way or the other. Nearly one third of the papers had disclosures, primarily regarding funding sources. After returning their evaluation of the research, all reviewers were surveyed.

Although 78 percent of the reviewers in the first group indicated they had read and understood the COI disclosures, and most also agreed that COIs are an important issue, the disclosures had no effect on reviewers' recommendations regarding papers' suitability for publication.

The study was led by Harvard's Leslie John, who earned her Ph.D. in Behavioral Decision Research at Carnegie Mellon in 2011. She said the findings likely are related to the ambiguity of the review process as it relates to COI disclosures.

"Reviewers aren't given any explicit guidance on how to correct for possible COI," said John, the Marvin Bower Associate Professor at Harvard Business School. "If a disclosure reveals a significant COI, should the reviewer recommend rejection or apply special scrutiny to methods? There currently isn't a consensus about how reviewers should respond to COI disclosures, nor any direction provided about how to do so."

John added that most of the reviewers reported that they strongly believed they could correct for the biasing influence of conflicts of interest if and when they were disclosed.

"Even experienced peer reviewers usually cannot articulate a specific and effective approach to follow once COI is suspected or declared," said Michael Callaham, AEM's editor-in-chief, emeritus chair of emergency medicine at UCSF and a senior author of the paper. "As a result, most reviewers do not change their assessments due to this information. Clearly the guidelines for reporting COI are not achieving their goal and need to be further studied and improved."

George Loewenstein, Carnegie Mellon's Herbert A. Simon University Professor of Economics and Psychology and a co-author of the study, noted that the paper is the latest in a long line of research showing that disclosure is an inadequate response to COIs that are rampant in medicine.

"The medical and academic establishments think they've dealt with the issues of COIs by including disclosures," Loewenstein commented. "This experiment provides further evidence that they haven't."
-end-
The study's fourth author, Andrew Marder, worked on the experiment as a senior statistician at Harvard Business School. The paper, "The Impact of Revealing Authors' Conflicts of Interests in Peer Review: A Randomised Controlled Trial," now appears online in the British Medical Journal.

Carnegie Mellon University

Related Emergency Medicine Articles:

Why is appendicitis not always diagnosed in the emergency department?
A new study examines the factors associated with a potentially missed diagnosis of appendicitis in children and adults in the emergency department.
NUS Medicine researchers can reprogramme cells to original state for regenerative medicine
Scientists from NUS Medicine have found a way to induce totipotency in embryonic cells that have already matured into pluripotency.
The cost of waiting in emergency departments
Wait times in US emergency departments are increasing. A new study published in Economic Inquiry indicates that prolonging the wait time in the emergency department for a patient who arrives with a serious condition by 10 minutes will increase the hospital's cost to care for the patient by an average of 6%, and it will increase the cost to care for moderately severe cases by an average of 3%.
Emergency medicine: Department-based intensive care unit improves patient survival rates
A new Michigan Medicine study found that implementing a dedicated emergency medicine department-based intensive care unit improved patient survival rates and lowered inpatient intensive care unit (ICU) admissions.
Emergency room or doctor's office?
A new study in the journal Heliyon, published by Elsevier, examines the relationship between the way individuals perceive and respond to threats (threat sensitivity) and where they most frequently seek medical care.
Gender-based salary gap persists among academic emergency medicine physicians
Although overall salaries for emergency physicians have increased over the past four years, and despite a call to end gender disparities in salary, men still make 18 percent more than women, and a $12,000 gender salary gap remains essentially unchanged.
Study reveals complementary medicine use remains hidden to conventional medicine providers
Research reveals that 1 in 3 complementary medicine (CM) users do not disclose their CM use to their medical providers, posing significant direct and indirect risks of adverse effects and harm due to unsafe concurrent use of CM and conventional medicine use.
Study of traditional medicine finds high use in Sub-Saharan Africa despite modern medicine
Researchers who have undertaken the first systematic review of into the use of traditional, complementary and alternative medicines (TCAM) in Sub-Saharan Africa found its use is significant and not just because of a lack of resources or access to 'conventional medicine'.
CU School of Medicine's Kenneth Tyler article in New England Journal of Medicine
Kenneth Tyler, M.D., the Louise Baum Endowed Chair in Neurology at the University of Colorado School of Medicine, is author of a review article about acute viral encephalitis in the current issue of The New England Journal of Medicine.
New program keeps elderly out of emergency
A medical program developed by emergency and palliative care clinicians at a large Australian hospital is seeing elderly aged care residents successfully treated at home.
More Emergency Medicine News and Emergency Medicine Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Climate Mindset
In the past few months, human beings have come together to fight a global threat. This hour, TED speakers explore how our response can be the catalyst to fight another global crisis: climate change. Guests include political strategist Tom Rivett-Carnac, diplomat Christiana Figueres, climate justice activist Xiye Bastida, and writer, illustrator, and artist Oliver Jeffers.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Speedy Beet
There are few musical moments more well-worn than the first four notes of Beethoven's Fifth Symphony. But in this short, we find out that Beethoven might have made a last-ditch effort to keep his music from ever feeling familiar, to keep pushing his listeners to a kind of psychological limit. Big thanks to our Brooklyn Philharmonic musicians: Deborah Buck and Suzy Perelman on violin, Arash Amini on cello, and Ah Ling Neu on viola. And check out The First Four Notes, Matthew Guerrieri's book on Beethoven's Fifth. Support Radiolab today at Radiolab.org/donate.