New method ranks quality of scientific journals by field

February 26, 2008

EVANSTON, Ill. --- Worldwide, the number of scientists is increasing as is the number of scientific journals and published papers, the latter two thanks in large part to the rise of electronic publishing. Scientists and other researchers are finding it more difficult than ever to zero in on the published literature that is most valuable to them.

Now, much like Google locates electronic information likely to be relevant and of quality to the user, a team of researchers from Northwestern University has developed a mathematical method to rank scientific journals according to quality, an approach that will help scientists locate high-impact research papers to read and to cite in their own papers. The rankings also should be useful to university administrators and funding agencies who must evaluate the quality of a researcher's work.

The team analyzed the citation data of nearly 23 million papers that appeared in 2,267 journals representing 200 academic fields and that spanned the years from 1955 to 2006; their analysis produced 200 separate tables of journal rankings by field. The results, including all the rankings, will be published online Feb. 27 in PLoS ONE, an international, peer-reviewed, open-access, online journal published by the Public Library of Science.

"Trying to find good information in the literature can be a problem for scientists," said Luís A. Nunes Amaral, associate professor of chemical and biological engineering in Northwestern's McCormick School of Engineering and Applied Science, who led the study. "Because we have quantified which journals are better, now we can definitively say that the journal in which one publishes provides reliable information about the paper's quality. It is important to grasp a paper's quality right away."

Amaral, Michael J. Stringer, lead author of the paper and a graduate student in Amaral's research group, and Marta Sales-Pardo, co-author and a postdoctoral fellow, developed methods to look at the enormous amount of published papers and to make sense of them. For each of the 2,267 journals, they charted the citations each paper received across a certain span of years and then developed a model of that data, which allowed the researchers to compare journals.

The researchers' model produced bell curves for the distribution of "quality" of the papers published in each journal. For each field, all the bell curves for the journals then were compared, which resulted in the journal rankings. The field of ecology, for example, had 36 journals, with Ecology ranked highest and Natural History ranked lowest.

"The higher a journal is ranked, the higher the probability of finding a high-impact paper published in that journal," said Amaral.

Amaral and his team found that the time scale for a published paper's complete accumulation of citations -- a gauge for determining the full impact of the paper -- can range from less than one year to 26 years, depending on the journal. Using their new method, the Northwestern researchers can estimate the total number of citations a paper in a specific journal will get in the future and thus determine -- right now -- the paper's likely impact in its field. This is the kind of information university administrators and funding agencies should find helpful when they are evaluating faculty members for tenure and researchers for grant awards.

"This study is just one example of how large datasets are allowing us to gain insight about systems that we could only speculate about before," said Stringer. "Understanding how we use information and how it spreads is an important and interesting question. The most surprising thing to me about our study is that the data seem to suggest that the citation patterns in journals are actually simpler than we would have expected."
-end-
The PLoS ONE paper is titled "Effectiveness of Journal Ranking Schemes as a Tool for Locating Information." Upon publication, the paper will be available at http://www.plosone.org/doi/pone.0001683.

Northwestern University

Related Data Articles from Brightsurf:

Keep the data coming
A continuous data supply ensures data-intensive simulations can run at maximum speed.

Astronomers are bulging with data
For the first time, over 250 million stars in our galaxy's bulge have been surveyed in near-ultraviolet, optical, and near-infrared light, opening the door for astronomers to reexamine key questions about the Milky Way's formation and history.

Novel method for measuring spatial dependencies turns less data into more data
Researcher makes 'little data' act big through, the application of mathematical techniques normally used for time-series, to spatial processes.

Ups and downs in COVID-19 data may be caused by data reporting practices
As data accumulates on COVID-19 cases and deaths, researchers have observed patterns of peaks and valleys that repeat on a near-weekly basis.

Data centers use less energy than you think
Using the most detailed model to date of global data center energy use, researchers found that massive efficiency gains by data centers have kept energy use roughly flat over the past decade.

Storing data in music
Researchers at ETH Zurich have developed a technique for embedding data in music and transmitting it to a smartphone.

Life data economics: calling for new models to assess the value of human data
After the collapse of the blockchain bubble a number of research organisations are developing platforms to enable individual ownership of life data and establish the data valuation and pricing models.

Geoscience data group urges all scientific disciplines to make data open and accessible
Institutions, science funders, data repositories, publishers, researchers and scientific societies from all scientific disciplines must work together to ensure all scientific data are easy to find, access and use, according to a new commentary in Nature by members of the Enabling FAIR Data Steering Committee.

Democratizing data science
MIT researchers are hoping to advance the democratization of data science with a new tool for nonstatisticians that automatically generates models for analyzing raw data.

Getting the most out of atmospheric data analysis
An international team including researchers from Kanazawa University used a new approach to analyze an atmospheric data set spanning 18 years for the investigation of new-particle formation.

Read More: Data News and Data Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.