UI study shows classification system for evaluating childhood hip disease may be unreliable

October 31, 1999

IOWA CITY, Iowa--A recent University of Iowa Health Care study is calling into question the reliability of a classification system that orthopaedic surgeons use to help them evaluate treatment outcomes in children with a hip condition known as Legg-Calve-Perthes Disease.

The system, developed in 1981, is called the Stulberg Classification System. The classification scheme predicts how a patient with the hip disease (often referred to as simply Perthes disease) will do over the long term.

"Physicians make decisions based on the system, but nobody had ever proven its reliability," said Stuart Weinstein, M.D., UI professor of orthopaedic surgery and the study's corresponding author. "Our findings indicate that the system has poor reliability. Consequently, the validity of any treatment decisions, outcome evaluations or epidemiological studies based on this system must be called into question."

Perthes disease is a condition that affects young children, most commonly between the ages of 4 and 10. It involves the loss of blood supply to the upper end of the femur, which eventually leads to deformation of the femoral head. If untreated, the disease may lead to degenerative arthritis. The cause of the disease is unknown, although roughly one in 12,000 children has the disease. The UI Hospitals and Clinics treats between 25 and 30 children with Perthes disease each year.

The Stulberg system establishes five patient outcome categories based on radiographs. Each category describes a different shape or relationship between the femoral head and the hip socket. Each category is associated with the potential for, and the onset of, degenerative joint disease. For example, children with category I hips are at low risk for early-onset degenerative disease, whereas children with category IV hips frequently develop arthritic changes in early adulthood. The system has also been used to evaluate the effectiveness of various treatments for Perthes disease.

For years, Weinstein noticed inconsistencies in outcome evaluations based on the system.

"What one surgeon might classify as being in the Stulberg III class, a colleague might decide falls into the Stulberg IV class," Weinstein explained. "This stems from the lack of objective, standardized language for evaluating the structures we're looking at on x-ray. This leads to problems in the clinic and in evaluating the literature."

Based on Weinstein's observation and the discovery that there was little published work concerning the reliability of the Stulberg system, a UI investigative team set out to measure how well surgeons agree with themselves and how well they agree with each other when using the system. The UI investigators also wanted to identify and quantify any sources of nonrandom error.

The UI study involved nine individuals representing three levels of experience --three staff pediatric orthopaedic surgeons, three senior residents and three junior residents. The nine raters independently used the Stulberg system to evaluate radiographs of 23 skeletally mature patients who had been managed for Perthes disease. None of the study raters were involved in the patients' care. Realizing that low reliability might be due to differing definitions, the investigators designed the study to involve an intervention where the team, following initial evaluations, discussed and standardized the conditions of each category. After seven to 10 days, the raters went back and re-evaluated the radiographs, as well as those of 22 additional patients. Researchers then compared reliability before and after the team discussion.

Regardless of the raters' experience levels, the UI study showed that each rater's individual reliability coefficient ranged between 71 and 92 before the team meeting and between 57 and 87 afterward. The reliability coefficient among raters ranged between 60 and 73 before the consensus-building session and between 65 and 74 after the session.

"Most interesting was the fact that the category distributions were significantly different before and after the team meeting," Weinstein said.

This indicates that the definitions used have a strong impact on the resulting classifications. Because of their sensitivity to differing definitions, this system is likely to be incomparable across physicians and studies when used to evaluate outcomes.

"The study results suggest that the Stulberg system is not a highly reliable tool for evaluating the radiographs of patients with Perthes disease at or after skeletal maturity," Weinstein said. "There were marked inter-rater variances among the nine physicians regardless of their experience level."

Weinstein said that the study results may be over-estimations of true reliability and that the reliability in actual clinical practice may be even lower.

"The participants knew that their ratings would be evaluated and compared with those assigned by their peers," Weinstein said. " In addition, because the participants were asked to review and memorize the Stulberg system and because they were guided by rating sheets, they probably were more knowledgeable about the system than the average orthopaedic surgeon is."

Although Weinstein questions the reliability of the current Stulberg system, he noted that it should not be entirely discounted.

"You don't want to just throw it out," he said. "Intuitively, it has some validity, but I think the whole classification scheme needs to be looked at. Perthes disease is a three-dimensional deformity, and these classifications are based on two-dimensional radiographs. In the future, we need to have other, more sophisticated methodologies to determine the anatomic outcome of the hip and relate it to the clinical outcome long term. More generally, the lesson of this work is not necessarily that the classification is unreliable, but that years of assuming otherwise may have misled investigators when reporting or making treatment decisions."

The UI findings appear in the September issue of the Journal of Bone and Joint Surgery.

University of Iowa

Related Children Articles from Brightsurf:

Black and Hispanic children in the US have more severe eczema than white children
A presentation at this year's virtual American College of Allergy, Asthma and Immunology (ACAAI) Annual Scientific Meeting reveals the disparities that exist for Black and Hispanic children when it comes to Atopic Dermatitis (AD), commonly known as eczema.

Black children with cancer three times less likely to receive proton radiotherapy than White children
A retrospective analysis led by investigators from Brigham and Women's Hospital has found racial disparities in the use of the therapy for patients enrolled in trials.

The Lancet Child & Adolescent Health: First Europe-wide study of children confirms COVID-19 predominately causes mild disease in children and fatalities are very rare
Children with COVID-19 generally experience a mild disease and fatalities are very rare, according to a study of 582 patients from across Europe published today in The Lancet Child & Adolescent Health journal.

Children not immune to coronavirus; new study from pandemic epicenter describes severe COVID-19 response in children
- While most children infected with the novel coronavirus have mild symptoms, a subset requires hospitalization and a small number require intensive care.

How many children is enough?
Most Russians would like to have two children: a boy and a girl.

Preterm children have similar temperament to children who were institutionally deprived
A child's temperament is affected by the early stages of their life.

Only-children more likely to be obese than children with siblings
Families with multiple children tend to make more healthy eating decisions than families with a single child.

Children living in countryside outperform children living in metropolitan area in motor skills
Residential density is related to children's motor skills, engagement in outdoor play and organised sports. that Finnish children living in the countryside spent more time outdoors and had better motor skills than their age peers in the metropolitan area.

Hispanic and black children more likely to miss school due to eczema than white children
In a study that highlights racial disparities in the everyday impact of eczema, new research shows Hispanic and black children are more likely than white children to miss school due to the chronic skin disease.

Children, their parents, and health professionals often underestimate children's higher weight status
More than half of parents underestimated their children's classification as overweight or obese -- children themselves and health professionals also share this misperception, according to new research being presented at this year's European Congress on Obesity (ECO) in Glasgow, UK (April 28-May 1).

Read More: Children News and Children Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.