How do studies get selected for publication?

June 05, 2012

The factors predicting the eventual impact of scientific research (i.e. high citation rates) were successfully identified in abstracts selected for presentation at the annual ESC Congress. Interestingly, predictors of publication in peer reviewed scientific journals differed markedly from those predicting later scientific impact, finds a recent study published online today in the European Heart Journal (EHJ).

The main factors predicting acceptance at the ESC Congress - in line with identified predictors of scientific impact - were the number of enrolled patients (>100) and prospective study design. In contrast, the factors predicting full-text publication in peer reviewed scientific journals were the institutional affiliation of the authors (i.e. university-affiliated institutions fared better than non university-affiliated institutions) and the gender of senior authors (males did better than females), finds the scientometric follow-up in the EHJ study.

The study, which provides a unique glimpse behind the "curtains" of the selection process of contemporary cardiovascular science at both congress and journal level, may indicate a hitherto unrecognised bias against non-academic institutions and female senior authors in journal study selection. "Neither gender nor institutional affiliation delineate scientific quality, which may cause some concern but requires further investigation," says the study first author Dr Stephan Winnik, a cardiologist in training at the University Hospital of Zurich, Switzerland.

The differences, he adds, may be due to the fact that abstracts submitted to the ESC Congress are subject to a double-blinded peer review process (where neither authors know reviewers nor vice versa); while the majority of scientific and medical journals (around 80%) use single-blinded peer review, where the reviewers know the identity of the authors, but the authors do not know the identity of reviewers. "Peer review has been and will be the sacred pillar of science, nonetheless our study suggests that introducing a double-blinded approach to journal review merits serious consideration," says Winnik.

The authors set out to identify factors that predicted the future success of studies as both abstracts and full-text articles, using the number of citations in the two years following publication as a surrogate for scientific quality. "The unique opportunity of this approach was the possibility to follow the same cohort of both accepted and rejected abstracts from first submission at a scientific meeting, through their full-text publication and subsequent impact after publication" says Winnik.

First, all 10,020 abstracts submitted to the 2006 World Congress of Cardiology in Barcelona (which combined the ESC and World Heart Federation congresses) were fed into a database; subsequently they were analysed according to whether they were accepted or rejected for presentation, and whether accepted studies were oral or poster presentations.

Next, a representative random selection of 10% of all submitted abstracts (n=1002) was analyzed according to a pre-specified set of variables, with the study cohort followed for five years for full-text publication and subsequent citation. Multivariate regression analyses were performed to identify predictors of scientific success at three major stages of the publication process - acceptance at the ESC Congress, subsequent full-text publication and the number of citations. The pre-specified variables included whether a study was clinical or basic, the type of institution where it took place, the study design, the number of patients enrolled, the field of clinical research and the gender of the first and last authors.

The following findings of the study are of note: "Given these data and other reports that blinded peer review does not only reduce bias but improve scientific quality, it can be speculated that blinded peer review merits consideration on a broader scale," state the authors in the discussion section of their article.

In an accompanying commentary Joseph Ross and Brahmajee Nallamothu, from Yale University School of Medicine, write, "The ESC did appear to choose wisely. Higher average peer reviewer ratings, and thus initial acceptance by the ESC, were associated with higher rates of publication and citation."

Nonetheless Ross and Nallamothu conclude that the fact that 70% of abstracts submitted to the 2006 World Congress of Cardiology remain unpublished five years later is "an unacceptable standard for scientific discourse".

Commenting on the study Professor Thomas F. Lüscher, editor of the EHJ and one of the authors of the article, says, "This study reassures us that the peer-review process of the ESC Congress works well, and provides an important quality control."

But the results at the full-text publication level are cause for concern, he adds, highlighting the finding that the type of institution where a study was performed predicted whether or not the study was published. "Whether this indicates that academic institutions are more successful in publishing their work due to experience and expertise or whether this reflects a bias against non-university institutions requires further study," says Lüscher, from the University Hospital of Zurich.

The possibility of blinding journal reviewers to both author identity and affiliation, he adds, warrants future consideration. At present, the EHJ follows the single-blinded approach. "We have discussed the possibility of changing to a double-blinded review process, but are aware that this would involve additional work and man-power, which is currently not available within our editorial office," he said.

Moreover, Lüscher adds, in the case of technically challenging projects, valuable insight can be gained from knowing the capabilities of the institutions involved.

The female gender of authors, says Lüscher, has the possibility to affect publication success in several ways. "Female scientists are more likely to work part-time and/or have other family obligations than their male counterparts, making finishing and publishing studies more difficult."

But there might also be a bias against females. "I have never experienced any such bias working with many journals, and additionally depending on the culture and name of authors gender is often not readily obvious."

European Society of Cardiology

Related Peer Review Articles from Brightsurf:

First published peer reviews of the WHO solidarity trials
Forthcoming COVID-19 research in Rapid Reviews: adjuvants could improve vaccine efficiency; what are the risks of reinfection; understanding autophagy could inspire new antivirals.

How that preprint about a 'more contagious strain' of coronavirus changed in peer review
On May 5, 2020, news broke about a reportedly more contagious variant of SARS-CoV-2 based on a preprint posted to bioRxiv.

Innovating the peer-review research process
A team of scientists led by a Michigan State University astronomer has found that a new process of evaluating proposed scientific research projects is as effective -- if not more so -- than the traditional peer-review method.

Monty Python's silly walk: A gait analysis and wake-up call to peer review inefficiencies
Fifty years ago, Monty Python's famous sketch, 'The Ministry of Silly Walks,' first aired.

Better science through peer review
The BioScience Talks podcast features discussions of topical issues related to the biological sciences.

Parental coaching adolescents through peer stress
During early adolescence, especially the transition to middle school, kids face a number of challenges both socially and academically.

Peer support reduces carer burden
In a world first, La Trobe University research has shown how peer-led support programs for family and friends who provide regular support to an adult diagnosed with a mental health condition can significantly improve carer well-being.

New research to explore technology needed for peer-to-peer 'free trade' in excess energy
People who generate their own power through solar panels and wind turbines may soon be able to decide where to distribute their excess energy, rather than back to the national grid.

Infants are more likely to learn when with a peer
Researchers at the University of Connecticut and University of Washington looked at the mechanisms involved in language learning among nine-month-olds, the youngest population known to be studied in relation to on-screen learning.

Potential gender bias against female researchers in peer review of research grants
Is peer review biased? Female health researchers who applied for grants from Canada's major health research funder were funded less often than male counterparts because of potential bias, and characteristics of peer reviewers can also affect the result, found a study in CMAJ (Canadian Medical Association Journal).

Read More: Peer Review News and Peer Review Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to