Special education professor advocates for steps to combat replication crisis in research

June 16, 2016

LAWRENCE -- Replication of scientific findings has been a cornerstone of validating research for generations, yet it happens so infrequently that many have claimed science is in a replication crisis. A University of Kansas special education professor has co-authored a study on replication, its effects on the field and students, and suggests a more dynamic approach to research could help address the paucity of replication.

Jason Travers, assistant professor of special education, was lead author of an article examining how the lack of research replications can negatively affect special education, and he argues that single-case experimental research design can complement group experiments to address the shortage. The study is the lead article in a special issue of the journal Remedial and Special Education -- edited by professors Kathleen Lane and Karrie Shogren of KU -- dedicated to the replication crisis.

Like nearly all academic fields, special education researchers face several challenges that make replicating research findings a challenging prospect. The article points out several challenges such as publishers' bias of novel findings, unrealistic expectations for researchers to publish a high quantity of original studies, a bias among the public and researchers for "striking" findings, lack of research funding for replication studies and many others. Travers co-authored the article with Bryan Cook of the University of Hawaii, William Therrien of the University of Virginia and Michael Coyne of the University of Connecticut.

While those problems are not unique to special education, they are troublesome because the field regularly develops interventions that educators will use for children with all manner of disabilities throughout the nation, Travers said. Without rigorous verification of previous findings, children could be subjected to interventions that are not truly effective or possibly even harmful.

For many years the standard approach to research testing and developing educational interventions has been group experimental designs. Special education is no different, but it is unique in a different respect.

"In special education we have a good deal of people who do research that utilizes the typical group paradigm. That's normal experimental work and a good chunk of what special education research looks like," Travers said. "The problem is that it can be very difficult to get a large group of participants for low-incidence disabilities, and group designs don't tell us much about who did and did not respond well to an intervention."

For example, autism, which is widely publicized and studied, is still technically a low-incidence disability. It can be very difficult to find 100 or more youths with autism to take part in a randomized group study. And even when possible, the range of variance among the youths -- some would be very highly functioning and fully communicative while others would not, not to mention age, cultural, socioeconomic and other differences -- make it very difficult to study the effectiveness of an intervention with a certain goal, such as improving communication.

Travers and co-authors therefore advocate for more use of a single-case research design, which could both lead to effective interventions and make replication less cumbersome.

A single-case experiment usually includes with one to five learners who are systematically introduced to an intervention and observed for changes in responding. The intervention may be removed and re-introduced to observe effects, such as the frequency of a response, to determine whether the intervention alters that response. The method can be used with a single learner at a time and many times over with individuals with varying levels of disability. The result is increased clarity about what types of individuals respond well or poorly to a certain intervention, and what variables increase or diminish responses for individuals as opposed to drawing conclusions from average responses by a group of individuals.

"In group experiments, causal effects are inferred. In single-case design, researchers explicitly demonstrate a causal relationship. They manipulate an independent variable to observe effects on dependent variables in ways that clarifies who does and doesn't respond to an intervention. We think this has some implications for the replication crisis," Travers said.

Increasing the number of single-case experiments could help increase the feasibility of replication by making it faster, more cost-effective and more practical to attempt. For example, special education research participants are most commonly students in public schools. In order to enroll them in a study, permission is needed from a university, school district, principals, teachers, parents and sometimes students themselves.

"Substantial money, time and other resources are often necessary to conduct a large group experiment of an intervention in applied settings," Travers said. "That alone affects how many studies can be conducted and therefore how many findings can be replicated before we can be confident we have a valid answer to a question."

The efficacy of single-case research design has long been established, yet it is not widely used. Medical and developmental disabilities researchers regularly use the method, but it is not as common in other areas of special education such as reading disorders and other disabilities, nor in other fields of scientific research. The method is sometimes confused with a qualitative case study, but Travers said single-case experiments differ in that variables are manipulated and effects are observed, as opposed to describing in detail what happens in a research setting.

Travers and colleagues argue that single-case experiments can work in a complementary way with group design research. Group design studies make valuable findings, but can be extremely difficult to replicate as they often take a skilled research team several years and millions of dollars to complete. By implementing a single-case design to replicate findings, researchers could not only do so more efficiently, they could add to the understanding of findings, even if they don't necessarily confirm the findings directly. Further, a collection of single-case experiments could inform researchers about a population from which samples could be drawn for large group studies. Conversely, results from large group studies can inform the selection of participants for single-case experiments to clarify what seems to influence responding.

Those advantages could also lend themselves to replicating research throughout the sciences, which could simultaneously help address the widespread lack of replication and bolster the validity of all manner of academic findings.

University of Kansas

Related Disabilities Articles from Brightsurf:

College students with disabilities at greater risk for substance abuse
College students with physical and cognitive disabilities use illicit drugs more, and have a higher prevalence of drug use disorder, than their non-disabled peers, according to a Rutgers study.

Asthma among children with developmental disabilities
How common asthma was among children with various developmental disabilities (including attention-deficit/hyperactivity disorder, autism spectrum disorder and vision, hearing or speech delay) was compared to children without disabilities in this survey study.

Children with developmental disabilities more likely to develop asthma
Children with developmental disabilities or delay are more at risk of developing asthma, according to a new study published in JAMA Network Open led by public health researchers at The University of Texas Health Science Center at Houston (UTHealth) as part of the Center for Pediatric Population Health.

Self-help groups empower caregivers of children with disabilities
Caregivers in low-income settings will be able to respond to the challenges of bringing up children with disabilities, thanks to a new model created by the University of East Anglia (UEA) and the Kenya Medical Research Institute (KEMRI).

Unintended pregnancy rates higher among women with disabilities, study says
Pregnancies among women with disabilities are 42% more likely to be unintended than pregnancies among women without disabilities, says a new report published in the journal Perspectives on Sexual and Reproductive Health.

More medical students are telling their schools about disabilities, and getting a response
The percentage of medical students who told their schools that they have a disability rose sharply in recent years, a new study shows.

The unpopular truth about biases toward people with disabilities
Needing to ride in a wheelchair can put the brakes on myriad opportunities -- some less obvious than one might think.

How to improve care for patients with disabilities? We need more providers like them
When it comes to patients with disabilities, the chance of getting a clinician 'like them' is extremely low, which may lead to patients' reluctance to seek care or follow prescribed interventions and treatments.

Progress to restore movement in people with neuromotor disabilities
A study published in the advanced edition of April 12, 2019 in the journal Neural Computation shows that approaches based on Long Short-Term Memory decoders could provide better algorithms for neuroprostheses that employ Brain-Machine Interfaces to restore movement in patients with severe neuromotor disabilities.

Certain physical disabilities may affect outcomes in kidney transplant recipients
Compared with kidney transplant recipients who did not report a disability, recipients with a visual disability were at higher risk of organ failure and recipients with a walking disability were at higher risk of early death.

Read More: Disabilities News and Disabilities Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.