Study shows how retractions significantly hurt scientists

September 06, 2017

Life scientists who have published papers that are retracted by journals subsequently suffer a 10 percent drop in citations of their remaining work, compared to similar but unaffected scientists, according to a new study by MIT researchers.

Examining hundreds of cases over a 30-year period, the research quantifies the extent that one discredited study -- whether an act of malfeasance or a sloppy piece of research -- has on the overall reputation of academic scientists.

"The question we're asking is: Do retractions trigger, at an individual level, something like an infection mechanism, where the retracted author is being punished and discredited for being dishonest or just incompetent?" says Alessandro Bonatti, an associate professor at the MIT Sloan School of Mangement and a co-author of a new paper detailing the study. "We find that yes, there is such a mechanism in place, and it operates through citations."

The study also finds that, in cases of clear misconduct, high-profile scientists who have a paper retracted experience an even larger drop -- 20 percent -- in the citations of their additional work.

The study adds to a growing literature on retractions and related problems in science, and suggests that the system of peer-review, while not perfect, does give people in the scientific community room to change their preferences about the quality of work presented to them.

As the authors write in the paper, the academic process of peer review may not provide "the optimal incentive system" in every regard, but the results do run against the "narrative that regards peer review as fundamentally undermined by ... forms of misconduct."

The paper, "The career effects of scandal: Evidence from scientific retractions," has just been published online by the journal Research Policy, and will appear in a print edition as well. The authors are Bonatti; Pierre Azoulay, the International Programs Professor of Management at MIT Sloan, who is the corresponding author; and Joshua L. Krieger PhD '17, an assistant professor at Harvard Business School, who worked on the project while completing his doctorate at MIT.

First retraction, then decline

The study looks at a large group of scientists who published papers, from 1977 until 2007, that were later retracted, and compares the citation rates of all their published works to the citation rates of a related group of scientists who never had papers retracted.

There are 376 scientists in the study who have had papers retracted and have collectively authored a total of 23,630 published papers in their careers. The control group of scientists without retractions includes 759 authors with a total of 46,538 published papers to their names.

There are two large methodological keys to the study. First, the control group of scientists was built by identifying researchers who had published papers in the same journal issues, and on similar topics, as the papers that were later retracted. This allowed the MIT researchers to be confident that they were studying two groups of scientists who were largely similar and could be expected to generate similar citation rates for their work.

Second, the 10 percent decline in citations is specifically measured against the "normal" trajectory of citations over time experienced by authors who have never had retractions. That is, academic papers generally do have a typical decline in citations over time. But the citation rates of still-valid papers published by researchers who subsequently had retractions dropped by 10 percent when compared to the expected long-term decline in citations rates that already exists.

"If you look at the time trend of citations to these papers, nothing [unusual] is happening until there is a retraction," Bonatti explains. "That's exactly what we're picking up. It's not that these are old papers that are getting obsolete, and people are citing newer stuff."

Instead, Bonatti adds, "Our data is consistent with a learning story." That is, the scientific community is reacting to new information, in the form of retractions, and re-adjusts its view about the value of the body of work of certain scholars, based on that.

"The mighty fall further"

The pattern the researchers discovered doubles when there is clear scientific misconduct, which is often announced by the journal making the retraction. As the researchers found, misconduct produces a subset of cases where the citation rates of other papers drop 20 percent, among scientists who had been among the top quartile of their peers in terms of citations.

"Once you're looking at retractions that involve misconduct, those are pretty good signs that something bad happened," Bonatti observes. "So when the signal is very clear, it doesn't matter how famous you were to begin with, you're going to be discredited. ... The mighty fall further, because they were standing taller to begin with."

The current study builds on prior work by Azoulay and Krieger, who in 2014 published a paper, along with two other co-authors, showing that retracted papers produced a 6 percent decline in citations for nonretracted papers in the same subfield of research.

As the scholars acknowledge, the new paper leaves open a number of related questions about retraction policy and the effects of retractions. Many journals publish retractions that present ambiguity about the reasons for their action, leaving outside observers unclear about the precise nature of the problem.

Moreover, as the authors write, they still "cannot say anything definitive" about the theoretical gains researchers receive from publishing dubious work; the study of retractions depends on suspect work being identified. It thus remains unclear how frequently scientists may be getting away with work that deserves retraction.
The research was supported, in part, by the National Science Foundation and the Sloan Foundation.

Additional background

Paper: The career effects of scandal: Evidence from scientific retractions

ARCHIVE: Modeling innovation

ARCHIVE: Should scientists handle retractions differently?

ARCHIVE: Scientists: This man has your number

ARCHIVE: Tracking the flow of knowledge

Massachusetts Institute of Technology

Related Peer Review Articles from Brightsurf:

First published peer reviews of the WHO solidarity trials
Forthcoming COVID-19 research in Rapid Reviews: adjuvants could improve vaccine efficiency; what are the risks of reinfection; understanding autophagy could inspire new antivirals.

How that preprint about a 'more contagious strain' of coronavirus changed in peer review
On May 5, 2020, news broke about a reportedly more contagious variant of SARS-CoV-2 based on a preprint posted to bioRxiv.

Innovating the peer-review research process
A team of scientists led by a Michigan State University astronomer has found that a new process of evaluating proposed scientific research projects is as effective -- if not more so -- than the traditional peer-review method.

Monty Python's silly walk: A gait analysis and wake-up call to peer review inefficiencies
Fifty years ago, Monty Python's famous sketch, 'The Ministry of Silly Walks,' first aired.

Better science through peer review
The BioScience Talks podcast features discussions of topical issues related to the biological sciences.

Parental coaching adolescents through peer stress
During early adolescence, especially the transition to middle school, kids face a number of challenges both socially and academically.

Peer support reduces carer burden
In a world first, La Trobe University research has shown how peer-led support programs for family and friends who provide regular support to an adult diagnosed with a mental health condition can significantly improve carer well-being.

New research to explore technology needed for peer-to-peer 'free trade' in excess energy
People who generate their own power through solar panels and wind turbines may soon be able to decide where to distribute their excess energy, rather than back to the national grid.

Infants are more likely to learn when with a peer
Researchers at the University of Connecticut and University of Washington looked at the mechanisms involved in language learning among nine-month-olds, the youngest population known to be studied in relation to on-screen learning.

Potential gender bias against female researchers in peer review of research grants
Is peer review biased? Female health researchers who applied for grants from Canada's major health research funder were funded less often than male counterparts because of potential bias, and characteristics of peer reviewers can also affect the result, found a study in CMAJ (Canadian Medical Association Journal).

Read More: Peer Review News and Peer Review Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to