Cognitive scores vary as much within test takers as between age groups making testing less valid

July 01, 2007

WASHINGTON -- How precise are tests used to diagnose learning disability, progressive brain disease or impairment from head injury" Timothy Salthouse, PhD, a noted cognitive psychologist at the University of Virginia, has demonstrated that giving a test only once isn't enough to get a clear picture of someone's mental functioning. It appears that repeating tests over a short period may give a more accurate range of scores, improving diagnostic workups.

The study is published in the July issue of Neuropsychology, which is published by the American Psychological Association (APA).

Salthouse gave 16 common cognitive and neuropsychological tests to evenly divided participants (90 in the first, 1600 in the second) into groups of ages 18-39, 50-59 and 60-97 years old. In both studies, the variation between someone's scores on the same test given three times over two weeks was as big as the variation between the scores of people in different age groups. It's as if on the same test, someone acted like a 20-year-old on a Monday, a 45-year-old the following Friday, and a 32-year-old the following Wednesday. This major inconsistency raises questions about the worth of single, one-time test scores.

"I don't think many people would have expected that the variability would be this large, and apparent in a wide variety of cognitive tests - not simply tests of speed or alertness," says Salthouse.

Psychologists frequently use tests of vocabulary, word recall, spatial relations, pattern comparison and the like to understand normal function and diagnose impairment. Experts use the scores to differentiate between diagnoses, detect changes in level of functioning or to give a diagnosis in the first place. Where scores fall relative to standardized cutoffs affects treatment, insurance, education plans and more. Yet the apparent fuzziness of one-time assessments could make it hard to tell whether someone is truly impaired, or truly improving or worsening, instead of showing normal short-term fluctuation.

Accordingly, Salthouse has come to believe that everyone has a range of typical performances, a one-person bell curve. Any given test will net a performance somewhere along that curve, as when a hitter's good and bad days are factored into a seasonal batting average. Some persons' scores would hew more closely to their average, but for those who have high internal variation, classification based on one assessment could be way off the mark.

Salthouse says it may be time to view cognitive abilities as a distribution of many potential levels of performance instead of as one stable short-term level. He proposes the use of a "measurement burst" procedure that bases understanding on several parallel assessments within a relatively short period. Results gained in this manner are likely to be more stable, offering a better basis for calibrating individual change.

Before any procedural updates, Salthouse says, "More will have to be learned about this phenomenon and the conditions under which it operates." Multiple assessments involve more time and expense but may be necessary, he notes, to distinguish short-term fluctuation from true ability level. In addition, psychologists would have to develop new test norms and truly equivalent versions of the same test.

Finally, Salthouse believes that measures of within-person variability could be a useful diagnostic marker in their own right. For example, he and other cognitive psychologists are discussing whether wilder fluctuations within one person's test scores are an early warning of mental decline.
-end-
Article: "Implications of Within-Person Variability in Cognitive and Neuropsychological Functioning for the Interpretation of Change," Timothy A. Salthouse, PhD, University of Virginia; Neuropsychology, Vol. 21, No. 4.

(Full text of the article is available from the APA Public Affairs Office and at http://www.apa.org/journals/releases/neu214401.pdf)

Dr. Salthouse can be reached at salthouse@virginia.edu.

The American Psychological Association (APA), in Washington, DC, is the largest scientific and professional organization representing psychology in the United States and is the world's largest association of psychologists. APA's membership includes more than 148,000 researchers, educators, clinicians, consultants and students. Through its divisions in 54 subfields of psychology and affiliations with 60 state, territorial and Canadian provincial associations, APA works to advance psychology as a science, as a profession and as a means of promoting human welfare.

American Psychological Association

Related Head Injury Articles from Brightsurf:

Soccer players' head injury risk could be reduced with simple adjustments to the ball
To reduce risk of soccer player head injury, a new study recommends preventing how hard a ball hits the head by inflating balls to lower pressures and subbing them out when they get wet.

Head to head comparison of five assays used to detect SARS-CoV-2 antibodies shows Siemens and Oxford assays met regulatory targets
New research being presented at the ESCMID Conference on Coronavirus Disease (ECCVID, online 23-25 September) shows that, in a head-to-head comparison of five tests used to detect SARS-CoV-2 antibodies (known as 'immunoassays'), an assay manufactured by Siemens and one developed by an academic partnership led by the University of Oxford had the most accurate results.

One in two homeless people may have experienced a head injury in their lifetime
People who are homeless experience a disproportionately high lifetime prevalence of traumatic brain injury (TBI), according to a new UBC-led study published today in The Lancet Public Health.

Global trial is first clear evidence that widely available drug reduces head injury deaths
A low cost and widely available drug could reduce deaths in traumatic brain injury patients by as much as 20%, depending on the severity of injury, according to a major study led by the London School of Hygiene & Tropical Medicine and published in The Lancet.

Microbleeds may worsen outcome after head injury
Using advanced imaging, researchers have uncovered new information regarding traumatic microbleeds, which appear as small, dark lesions on MRI scans after head injury but are typically too small to be detected on CT scans.

Protein tangles linked with dementia seen in patients after single head injury
Scientists have visualized for the first time protein 'tangles' associated with dementia in the brains of patients who have suffered a single head injury.

Head injury effects halted by xenon gas, finds first ever life-long study in mice
Following traumatic brain injury (TBI), xenon prevented early death, improved long-term cognition, and protected brain tissue in mice in a new study.

Mental health disorders common following mild head injury
A new study reveals that approximately 1 in 5 individuals may experience mental health symptoms up to six months after mild traumatic brain injury (mTBI), suggesting the importance of follow-up care for these patients.

One type of brain cell might hold key to inflammation after head injury
By eliminating one type of immune cell in the brain, researchers were able to erase any evidence of inflammation following traumatic brain injury, according to a new study from The Ohio State University.

Youth TBI laws promote head injury evaluation in emergency department
A new study from researchers in the Center for Injury Research and Policy at Nationwide Children's Hospital examined the effectiveness of state youth TBI laws by looking at sports and recreation mild TBI (mTBI)-related emergency department (ED) visits for children ages 5 to 18 years before and after TBI legislation was enacted in each state.

Read More: Head Injury News and Head Injury Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.