Strong scientific peer review leads to better science and policy formation

January 19, 2011

The current Special Issue of Technology & Innovation, Proceedings of the National Academy of Inventors ™, (, now available on-line at: , is focused on the history, process and practice of scientific peer review, with several articles aimed at assessing scientific peer review within the federal government and peer review's relationship to federal policy formation.

According to A. Alan Moghissi, president of the nonproft Institute for Regulatory Science, and Michael S. Swetnam, of the Potomac Institute for Policy Studies, some confusion exists over what peer review is and how it should be used in policy decision making.

For Moghissi and Swetnam, advancements in science and technology drive an increase in the need for critical evaluation by the independent peers of those who create the advancements.

"Reviewers must be qualified and independent," they wrote. "Once more, review criteria must be identified, the process must be transparent, and over reliance on specific peer reviewers must be avoided. Too, institutional, intellectual and personal conflicts of interest must also be avoided."

Maintaining the integrity and improving peer review in the federal government was a key issue for Elmer Yglesias of the Science and Technology Policy Institute, Institute for Defense Analyses, who wrote that "there are indications that the (federal peer review) system may be over stretched and prone to error."

"Unprecedented high numbers of proposals and low funding rates may lead one to wonder whether the (federal) peer review system is up to the challenge," said Yglesias. "Indications are that it is over stretched."

According to Yglesias, NSF surveys indicate that reviewer's workload has increased and the number of reviewers per proposal has been declining, making funding decisions susceptible to error. The best solution to ensure review's integrity, he wrote, may lie in an Internet-based calibration system.

Nowhere may the process of peer review be more important than in determining what should be funded in military medical research. Dr. John F. Glenn, principal assistant for research and technology for the U.S. Army Medical Research and Materiel Command, made the point that candidate products may include prototypes for vaccines, drugs, diagnostics and other medical devices, information products, such as for training and behavioral interventions, as well as human performance information and information on a wide range of hazards.

"A distinctive feature of the core research programs is that they are end product-focused," wrote Glenn. "This leads to three separate roles for peer review - evaluation of research project quality, evaluation of research program quality, and independent validation of research products."

Important to Glenn is determining that a peer review process can validate that end products have met their objectives. He noted that the U.S. Food and Drug Administration's review process employs both internal and external scientific review panels and that the FDA's process is accepted as a substitute by the Department of Defense.

"Unfortunately, there is no comparable, consistently employed peer review process for the validation of medication information or clinical practice guidelines," wrote Glenn. "Such processes are needed as they are intrinsic for evidence-based medicine."

Not only does good peer review help validate a projects science, wrote lead author E. Melissa Kaine, MD, Captain, Medical Corps, U.S. Navy, it can also foster innovation.

According to Dr. Kaine, the Congressionally Directed Medical Research Program (CDMRP) is comprised of 18 individual programs seeking to eradicate diseases and support the 'warfighter.' The CDMRP emphasizes innovative, high-risk, high-gain research and employs a two-tier process to evaluate scientific merit, innovation and impact, followed by an external review. These processes, she said, support the mission to fund innovative research. While 'innovative' may be difficult to define, research that is transformative, paradigm shifting and that which represents more than an incremental advance in existing knowledge, serves as definitive.

One innovative aspect of the CDMRP review process is the use of layperson reviewers in addition to scientific experts.

"Consumer reviewers are included and involved at every stage of the funding cycle," she noted. "These reviewers are focused on the impact of the proposed project and provide valuable insight into a projects potential to result in positive outcomes."

Finally, the National Science Foundation (NSF) historian, Marc Rothenberg, reported on the history of peer review at the 60-year-old agency.

"NSF has had to consider 'broader impact' in addition to 'good science' in its mission to support basic research in the wide spectrum of science and engineering disciplines," wrote Rothenberg. "NSF has also fought a continuing battle to clarify peer review criteria and to persuade peer communities to use the criteria as set down."

Rothenberg added that the trend at NSF since the 1960s has been to reduce the number of criteria, but to broaden the definition of those remaining.
The National Academy of Inventors ™ recognizes and encourages inventors who have a patent issued from the U.S. Patent and Trademark Office, enhances the visibility of university technology and academic innovation, encourages the disclosure of intellectual property, educates and mentors innovative students, and encourages the translation of the inventions of its members to benefit society.

The editorial offices of Technology & Innovation are located at the University of South Florida, Office of Research and Innovation, 3702 Spectrum Blvd., Suite 175, Tampa, Florida, 33612. Tel: (813) 974-3181. Email

News Release by Randolph Fillmore, Florida Science Communications,

University of South Florida (USF Health)

Related Innovation Articles from Brightsurf:

Food system innovation -- and how to get there
Food production has always shaped the lives of humans and the surface of the Earth.

What is the best way to encourage innovation? Competitive pay may be the answer
Economists and business leaders agree that innovation is a major force behind economic growth, but many disagree on what is the best way to encourage workers to produce the 'think-outside-of-the-box' ideas that create newer and better products and services.

Innovation is widespread in rural areas, not just cities
Conventional measures of innovation suggest that only big cities foster new ideas, but a more comprehensive measure developed at Penn State shows that innovation is widespread even in rural places not typically thought of as innovative.

Scaling up search for analogies could be key to innovation
Investment in research is at an all-time high, yet the rate of scientific breakthroughs isn't setting any records.

Why you should be concerned about Oprah Winfrey when introducing an innovation
New research by Bocconi University's Paola Cillo and Gaia Rubera with Texas A&M's David Griffith asserts that the reaction of large individual investors to innovation is an important component of stock returns, their reaction to innovation depends on their national culture, and there is a way to segment large individual investors and pitch innovation to them accordingly.

Responsible innovation key to smart farming
Responsible innovation that considers the wider impacts on society is key to smart farming, according to academics at the University of East Anglia (UEA).

Pillars of academic innovation
Highlights from the Sixth Annual Conference of the National Academy of Inventors, including high-tech solutions to combat child pornography and radicalization materials; groundbreaking programs to promote STEM major retention; and new materials for wearable technology.

Universities drive innovation in the classroom
The current issue of Technology and Innovation, Journal of the National Academy of Inventors ® (19.2) examines innovation from the university perspective, highlighting what the most innovative institutions and educators worldwide are doing to prepare future engineers and industry leaders to effectively manage IP to grow their companies and the global economy as a whole.

How universities are fostering innovation and entrepreneurship
Technology and Innovation 19.1 zeroes in on innovation and entrepreneurship, with a special focus on what universities are currently doing to foster growth in those areas both for their success and the success of the communities and regions to which they are connected.

Shaping the future of health innovation
Future advances in healthcare will be aided by a new £10 million facility -- the National Institute for Health Research Innovation Observatory based at Newcastle University, UK.

Read More: Innovation News and Innovation Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to