Pediatric clinical studies appear prone to bias, Hopkins review shows

July 12, 2010

A Johns Hopkins review of nearly 150 randomized controlled trials on children -- all published in well-regarded medical journals -- reveals that 40 to 60 percent of the studies either failed to take steps to minimize risk for bias or to at least properly describe those measures.

A report of the team's findings in the August issue of Pediatrics shows that experimental trials sponsored by pharmaceutical or medical-device makers, along with studies that are not registered in a public-access database, had higher risk for bias. So were trials that evaluate the effects of behavioral therapies rather than medication, the report states.

"There are thousands of pediatric trials going on in the world right now and given the risk that comes from distorted findings, we must ensure vigilance in how these studies are designed, conducted and judged," says lead investigator Michael Crocetti, M.D., M.P.H., a pediatrician at Johns Hopkins Children's Center. "Our review is intended as a step in that direction."

Considered the gold standard of medical research, the hallmark of double-blind randomized controlled trials (RTC) is a design that rules out or accounts for actual or potential bias. Results of such studies, when peer-reviewed and published in reputable medical journals, can influence the practice of medicine and patient care. A poorly designed or executed trial can therefore lead researchers to erroneous conclusions about the effectiveness of a drug or a procedure.

Citing the degree of bias risk in the studies they reviewed, the researchers caution pediatricians to be critical readers of studies, even in highly respected journals.

The investigators advise that when reading a report on a trial, pediatricians should not merely look at the bottom line but ask two essential questions: How did the researchers reach the conclusion? and Was their analysis unbiased?

Doctors should apply "smell tests," common sense and skeptical judgment about whether the conclusions fit the data, especially when a study boasts dramatic effects or drastic improvement.

Crocetti and colleagues used the Cochrane Collaboration tool, which assesses risk for bias along six critical aspects including randomization -- randomly assigning patients to different treatments -- and masking, the degree to which neither the patient nor the doctor knows which group of patients is receiving an active drug or intervention versus a placebo.

Investigators say that by analyzing each clinical trial along these and four other dimensions, the Cochrane Collaboration can answer what are perhaps the most important questions in medical research: How strong is the causal relationship between the therapy and the effect? and How valid are the conclusions made about the effect of the therapy?

Overall, 41 percent of the 146 trials in the review had improper or poorly described randomization techniques. Industry-funded trials were six times more likely to have high risk for biased randomization than government-funded trials or those funded by nonprofit organizations. And past research, the investigators point out, has shown that industry-funded trials are four to five times more likely to recommend an experimental drug.

"Industry funding is an important driver of medical discovery, but it is critical for investigators involved in such trials to ensure not only that the studies are conceived and executed cautiously with minimum risk for bias, but that any precautions taken against bias are also reported transparently," Crocetti says.

Trial registration -- and the transparency commitment it reflects -- is a key step in reducing bias or its influence, the researchers say. In their evaluation, registered trials were nearly 70 percent more likely to have robust randomization than non-registered trials, probably because the registration process itself forces researchers to answer many questions related to trial design and execution. Therefore, the investigators say, registering pediatric trials in the public domain will not only increase transparency but, in the long run, improve the validity of their results. In 2005, the International Committee of Medical Journal Editors called for registering all medical trials involving human subjects with www.ClinicalTrials.gov, a massive public repository of more than 70,000 trials from all over the world. However, less than 60 percent of the pediatric trials in the review were registered there.

The researchers also found that most of the trials (57 percent) either failed to use proper techniques that ensure anonymity or "blinding" to the type of treatment a patient gets, or they failed to clearly describe these techniques. The technique, called allocation concealment, ensures that neither the researcher nor the patient can guess which treatment they will get. The method also helps ensure that the treatment of one subject will not reveal to either scientists or the patients clues about the treatment of the next subject. Trials involving behavioral therapies were four times more likely to have this problem.

Overall, nearly 20 percent of the trials used improper masking techniques to ensure that neither the patient nor the researchers know which treatment went to which patient.

The 146 trials in the review appeared between 2007 and 2008 in the five leading pediatric journals -- Pediatrics, Pediatric Infectious Diseases, Journal of Pediatrics, Archives of Pediatrics and Adolescent Medicine and the Journal of the American Academy of Child and Adolescent Psychiatry -- and the three highest-ranked general medical journals -- JAMA, The New England Journal of Medicine and The Lancet.
-end-
Co-investigators on the research: Roberta Scherer, Ph.D., of the Johns Hopkins Bloomberg School of Public Health and Diane Amin, PA-C., of Hopkins.

Related on the Web:
Johns Hopkins Children's Center
http://www.hopkinschildrens.org

Michael Crocetti, M.D., M.P.H.
http://www.hopkinschildrens.org/staffDetail.aspx?id=3194

Parents Favor E-mail Access to Their Pediatricians
http://www.hopkinschildrens.org/Parents-Favor-Email-Access-to-Their-Pediatricians.aspx

Pediatrics
http://www.pediatrics.org

Johns Hopkins Medicine

Related Medical Research Articles from Brightsurf:

Patients say ask before using medical records for research
A new study led by Michigan Medicine researchers finds that even when patients understand the overall benefit to society, they still want to be able to give consent at least once before their de-identified data is used for research.

Most patients willing to share medical records for research purposes
In a survey, UC San Diego researchers report most patients are willing to share medical records for research purposes, with a few caveats.

Tax hurts investment in medical device research and development
New Iowa State University research shows companies cut funding for research and development in response to a tax imposed on medical devices as part of the Affordable Care Act.

Centralized infrastructure facilitates medical education research
The Council of Academic Family Medicine Educational Research Alliance has enabled a large number of research teams to conduct meaningful scholarship with a fraction of the usual time and energy.

Sex, gender, or both in medical research
Only a minority of medical studies take sex and gender into account when analyzing and reporting research results.

Research!America to honor medical and health research advocacy leaders
Research!America's 21st annual Advocacy Awards will honor outstanding advocates for research whose contributions to health and medicine have saved lives and improved quality of life for patients worldwide.

Ohioans say it is important for the state to lead in education and medical research
An overwhelming majority of Ohio residents say it is important for the state to be a leader in education (89 percent) and in medical and health research (87 percent), according to a state-based public opinion survey commissioned by Research!America.

Medical research influenced by training 'genealogy'
By analyzing peer-reviewed scientific papers that examined the effectiveness of a surgical procedure, researchers at University of California, San Diego School of Medicine provide evidence suggesting that the conclusions of these studies appear to be influenced by the authors' mentors and medical training.

Diversity in medical research is a long way off, study shows
Despite Congressional mandates aimed at diversifying clinical research, little has changed in the last 30 years in both the numbers of studies that include minorities and the diversity of scientists being funded, according to a new analysis by researchers at UCSF.

Research!America to honor leaders in medical and health research advocacy
Research!America's 20th annual Advocacy Awards will honor exceptional advocates for research whose achievements in their fields have brought hope to patients worldwide.

Read More: Medical Research News and Medical Research Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.