Structured reporting software creates less complete and accurate radiology reports than free text

November 16, 2009

WINSTON-SALEM, N.C. - As many software companies work to create programs that will give uniform structure to the way radiological test results are reported, a new study by researchers at Wake Forest University School of Medicine shows that such a system does not improve, but rather decreases the completeness and accuracy of the reports.

The study, published recently in Radiology, compared the accuracy and completeness of reporting test results in a free text, narrative format versus using standardized words and phrases from a pull-down menu (structured reporting).

"This research is our attempt to evaluate a new technology that is a pretty hot topic in medicine right now and has been for a few years," said Annette J. Johnson, M.D., M.S., an associate professor of radiology and lead investigator on the study. "Since radiology began, we have been creating our reports in a free text, narrative format. The rationale behind efforts to change this format is that all of the reports that we create could potentially be a very useful data base for clinical care and research if they were standardized."

Standardization would mean that key content could be accessed through automated means, by computer systems, rather than requiring a human being to read the report and manually sift through narrative comments to try to find and categorize key content, Johnson said.

"This type of standardization is a very appealing idea, but we did not have data regarding what effect structured systems like this might have on individual report quality until now," she added. "It turns out that a structured reporting system actually decreases the accuracy and especially the completeness of reports, which is the opposite of what we expected."

This study provides the only known data about the proposed program's effect on the quality of radiological reports, Johnson said.

Currently, she explained, a physician might send a patient who is experiencing weakness in his arm, for example, for a head computed tomography (CT) scan to rule out concerns about a stroke. A radiologist then reads the CT scan and reports what she sees.

"I might say, 'There's no evidence of hemorrhage,' or any other variety of wording to convey that idea. I could say, 'no bleeding' or 'no hematoma' or 'no hyperdensity,' all of which mean the same thing," Johnson said. "In a structured system, I would choose from a list of standardized phrases with certain specific terms available in a dropdown menu, such as 'No presence of acute stroke.' Standardization seems simple, but it's not always easy or what we commonly do in medicine. The theory is that structured reporting would make the reports intrinsically better because we'd all be using the same ideas recorded in the same verbiage instead of using numerous different ways to say 'blood.' Right now, several people reading a scan may all agree that they see the same thing, but each individual will say it in a different way."

For the study, the researchers tested such a structured reporting system on two groups of residents. Each individual in both groups was given the same set of 25 brain magnetic resonance imaging (MRI) scans along with a video of a staff physician's interpretation of the MRI scans, and was asked to report the interpretation using the free text narrative they were familiar with using. Four months later, the same set of MRIs was given separately to each individual again. Half of the residents were asked to report their observations in the free text narrative form as they had the first time. The other half were asked to create their reports using the structured reporting software, which listed standardized sentences and phrases describing different findings to choose from.

"We thought that the structured reporting group would make better reports," Johnson said. "However, the reports created using structured reporting software were actually substantially less complete and a little less accurate compared to the reports made by the same residents in free text four months earlier and with the other group of residents who used free text both times."

The company that made the specific structured reporting software used for the study is no longer in existence, Johnson said, but there are still many more software companies very focused on creating these programs and finding ways to structure reports. These companies, and the physicians who choose to use structured reporting systems, should strongly consider how the software is going to affect the quality of real patient records, and all software should be specifically tested for quality effects before being implemented, she said.
-end-
The study was funded in part by the General Electric-Association of University Radiologists Radiology Research Academic Fellowship (GERRAF).

Media Relations Contacts: Jessica Guenzel, jguenzel@wfubmc.edu, (336) 716-3487, or Bonnie Davis, bdavis@wfubmc.edu, (336) 716-4977.

Wake Forest University Baptist Medical Center (www.wfubmc.edu) is an academic health system comprised of North Carolina Baptist Hospital, Brenner Children's Hospital, Wake Forest University Physicians, and Wake Forest University Health Sciences, which operates the university's School of Medicine and Piedmont Triad Research Park. The system comprises 1,056 acute care, rehabilitation and long-term care beds and has been ranked as one of "America's Best Hospitals" by U.S. News & World Report since 1993. Wake Forest Baptist is ranked 32nd in the nation by America's Top Doctors for the number of its doctors considered best by their peers. The institution ranks in the top third in funding by the National Institutes of Health and fourth in the Southeast in revenues from its licensed intellectual property.

Wake Forest Baptist Medical Center

Related Radiology Articles from Brightsurf:

Data science pathway prepares radiology residents for machine learning
A recently developed data science pathway for fourth-year radiology residents will help prepare the next generation of radiologists to lead the way into the era of artificial intelligence and machine learning (AI-ML), according to a special report.

Radiology research funding has increased -- still no association with citation rate
According to ARRS' AJR, almost half of the research articles published in AJR, Radiology, and European Radiology declared funding -- a proportion that has increased from 17% of articles in 1994 and 26.9% published between 2001 and 2010.

Unread second-opinion radiology reports waste health care resources
According to ARRS' American Journal of Roentgenology (AJR), 537 of 4,696 second-opinion reports--11.4%, likely underestimated, too--were not read by a clinician.

Radiology practices struggle to survive amid COVID-19
Private radiology practices have been especially hard hit by the COVID-19 pandemic, and the steps they take to mitigate the impact of the pandemic on their practice will shape the future of radiology, according to a special report from the Radiological Society of North America (RSNA) COVID-19 Task Force.

Social media and radiology -- The good, the bad, and the ugly
Radiologists examine social media and report #SoMe can be useful in education, research, mentoring and career development.

Experts stress radiology preparedness for COVID-19
Today, the journal Radiology published the policies and recommendations of a panel of experts on radiology preparedness during the coronavirus disease (COVID-19) public health crisis.

Radiology: Cardiothoracic Imaging publishes special report on vaping
Radiology: Cardiothoracic Imaging has published a special report on lung injury resulting from the use of electronic cigarettes, or 'vaping.' Researchers aim to raise awareness among radiologists and other medical professionals on how to identify e-cigarette or vaping product use-associated lung injury.

Radiology organizations publish statement on ethics of AI in Radiology
Experts in the use of artificial intelligence (AI) in radiology, from many of the world's leading radiology, medical physics and imaging informatics groups, today published an aspirational statement to guide the development of AI in radiology.

Rate of radiology resident recognition of non-accidental trauma
Radiology residency programs nationwide are not adequately teaching residents to accurately recognize and report child abuse, according to a study to be presented at the ARRS 2019 Annual Meeting, set for May 5-10 in Honolulu, Hawaii.

Radiology publishes roadmap for AI in medical imaging
In August 2018, a workshop was held at the National Institutes of Health (NIH) in Bethesda, Md., to explore the future of artificial intelligence (AI) in medical imaging.

Read More: Radiology News and Radiology Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.