New statistical model improves the predictive power of standardized test scores

November 19, 2019

A standout essay, high grade point average and stellar standardized test scores are sometimes not enough for college admissions.

The ongoing college admission scandal underscores how influential a standardized test score has become. A test administrator is now cooperating with the investigation into other parents who paid to have their children's test scores fixed.

College admissions decisions use standardized test scores as a predictor of how well an applicant will do in college. But what if there were a better way to predict learning - one that did not rely on a single, high-stakes test?

Researchers from the Arizona State University and the University of Denver have devised a way to predict academic performance that is three times more predictive than a single standardized assessment. The research team developed and validated a statistical model that uses readily available test scores to predict future academic performance. The study will be published in Multivariate Behavioral Research.

"Everyone is affected by testing at some point - tests are used to make high-stakes decisions about admissions to schools and sometimes even job placement - and the model we developed captures what is going on in the data and predicts future performance better than existing methods," said Daniel McNeish, assistant professor of psychology at ASU and first author on the paper.

Current ability does not always predict future learning

The stated purpose of many standardized tests is a one-time assessment, not to inform long-term performance. These tests are sometimes used to predict the future performance of anyone who takes the test, but few tests actually do this well, said Denis Dumas, who is an assistant professor at the University of Denver and second author on the paper. The idea that a single test can fail to adequately measure a student's future learning potential is not a new one: The sociologist, historian and civil rights activist W.E.B. DuBois raised it almost a century ago.

"Test scores from a single time point give a good snapshot of what someone knows at the time of testing, but they often are incapable of providing information about the potential to learn," added Dumas. "Test scores are frequently used to indicate how much a person might benefit from future education, like attending college, but this concept is completely different from how much the test taker knows right now."

To develop the model, the research team took inspiration from the work of an Israeli psychologist named Reuven Feuerstein who tested children survivors of the Holocaust for school and grade-level placement. Grade-level assignments based on one test score were often too low, so Feuerstein developed a testing system called dynamic assessment that used several test scores collected over time to measure children's capacity to learn, instead of their current level of knowledge. Dynamic assessment is labor-intensive and is difficult to implement on a large scale. The research team solved that problem by leveraging advances in mathematical models and computing power to create a new method, which they call a dynamic measurement model.

Connecting the dots

The dynamic measurement model uses a series of test scores to predict future learning capacity. The model fits a curve through the test scores over time, which usually looks like a sideways letter "J" and is ofted called a "learning curve." The points on the learning curve represent the amount of current knowledge, and the maximum or ceiling of the curve is the learning potential. Using standardized test scores from kindergarten through eighth grade, the team recently showed the dynamic measurement model could fit the learning curve and predict learning potential.

The research team wanted to know how far out the model could predict learning potential and thus how accurate it actually was. They used three datasets that originated from the Institute of Human Development at the University of California, Berkeley. The datasets include test scores from participants starting when they were 3 years old in the 1920s and 1930s. The participants were studied for decades, until they were in their 50s, 60s, and 70s.

Because most standardized testing happens in school, the research team used the dynamic measurement model to fit the test scores from when the UC Berkeley participants were aged 20 and younger. The team predicted the future learning potential of each participant by having the model finish the curve. Then, they compared the actual test scores at ages 50-70 years to what the model predicted.

"The dynamic measurement model captured three times the variance as other methods, including single time-point test scores. In other words, our model predicted the later scores -- an individual's realized learning potential -- three times better," McNeish said. "Students are tested so frequently now to gauge their progress, but having multiple scores per student can serve a purpose beyond gauging progress. They can be combined into a single learning potential score to improve predictions of where people's skills and abilities are predicted to end up in the future if they maintain the same trajectory."

Harnessing the potential of standardized testing

Using dynamic measurement modeling to predict the future learning potential of students does not require changes in policy or new tests. The test scores needed for the model already exist and are available because of the passage of the No Child Left Behind Act and Every Student Succeeds Act.

"Dynamic measurement modeling does not require a specialized computer to run and does not take much longer than standard statistical models used in this area," McNeish said. "Logistically, all the pieces are there to implement it tomorrow."

The research team is currently working on developing software to disseminate the dynamic measurement model.
-end-
ASU's Kevin Grimm, professor of psychology, also contributed to this study.

Arizona State University

Related Human Development Articles from Brightsurf:

'Monster tumors' could offer new glimpse at human development
Finding just the right model to study human development--from the early embryonic stage onward--has been a challenge for scientists over the last decade.

How herpes infection may impair human fetal brain development
Three cell-based models shed light on how herpes simplex virus type 1 (HSV-1) infection, which can spread to the fetal brain during pregnancy, may contribute to various neurodevelopmental disabilities and long-term neurological problems into adulthood, according to a study published October 22, 2020 in the open-access journal PLOS Pathogens by Pu Chen and Ying Wu of Wuhan University, and colleagues.

New platform gauges effects of plastic nanoparticles on human development and health
A study released today in STEM CELLS outlines a new platform researchers designed that allowed them to investigate the potentially harmful effects of microplastics and nanoplastics.

Controlled human infection models and SARS-CoV-2 vaccine development
Infecting volunteers with COVID-19 may provide valuable insights for future rounds of vaccine testing, but would require very strict controls and is unlikely to advance the current slate of vaccines in advanced development, argues a group of infectious disease experts.

New method provides unique insight into the development of the human brain
Stem cell researchers at Lund University in Sweden have developed a new research model of the early embryonic brain.

UCLA scientists create first roadmap of human skeletal muscle development
An interdisciplinary team of researchers at the Eli and Edythe Broad Center for Regenerative Medicine and Stem Cell Research at UCLA has developed a first-of-its-kind roadmap of how human skeletal muscle develops, including the formation of muscle stem cells.

First in-human study of drug targeting brain inflammation supports further development
MW189 blocks abnormal inflammation in the brain that is known to contribute to injury- and disease-induced neurologic impairments in a number of acute and chronic brain disorders.

Reconstructing the clock of human development
Researchers used iPS cells to reconstructed the human 'segmentation clock,' a key point in early embryonic development that determines how the body gets segmented.

Component of human breast milk enhances cognitive development in babies
CHLA investigators show that early exposure to a carbohydrate found in breast milk, called 2'FL, positively influences neurodevelopment.

Organoids open window into development of human forebrain
Brain region-specific organoids have allowed researchers to peer inside the complex programming of human forebrain development, a process once inaccessible to molecular study.

Read More: Human Development News and Human Development Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.