AI algorithms detect diabetic eye disease inconsistently

January 05, 2021

Diabetes continues to be the leading cause of new cases of blindness among adults in the United States. But the current shortage of eye-care providers would make it impossible to keep up with demand to provide the requisite annual screenings for this population. A new study looks at the effectiveness of seven artificial intelligence-based screening algorithms to diagnose diabetic retinopathy, the most common diabetic eye disease leading to vision loss.

In a paper published Jan. 5 in Diabetes Care, researchers compared the algorithms against the diagnostic expertise of retina specialists. Five companies produced the tested algorithms - two in the United States (Eyenuk, Retina-AI Health), one in China (Airdoc), one in Portugal (Retmarker), and one in France (OphtAI).

The researchers deployed the algorithm-based technologies on retinal images from nearly 24,000 veterans who sought diabetic retinopathy screening at the Veterans Affairs Puget Sound Health Care System and the Atlanta VA Health Care System from 2006 to 2018.

The researchers found that the algorithms don't perform as well as they claim. Many of these companies are reporting excellent results in clinical studies. But their performance in a real-world setting was unknown. Researchers conducted a test in which the performance of each algorithm and the performance of the human screeners who work in the VA teleretinal screening system were all compared to the diagnoses that expert ophthalmologists gave when looking at the same images. Three of the algorithms performed reasonably well when compared to the physicians' diagnoses and one did worse. But only one algorithm performed as well as the human screeners in the test.

"It's alarming that some of these algorithms are not performing consistently since they are being used somewhere in the world," said lead researcher Aaron Lee, assistant professor of ophthalmology at the University of Washington School of Medicine.

Differences in camera equipment and technique might be one explanation. Researchers said their trial shows how important it is for any practice that wants to use an AI screener to test it first and to follow the guidelines about how to properly obtain images of patients' eyes, because the algorithms are designed to work with a minimum quality of images.

The study also found that the algorithms' performance varied when analyzing images from patient populations in Seattle and Atlanta care settings. This was a surprising result and may indicate that the algorithms need to be trained with a wider variety of images.
-end-
This study was supported by NIH/NEI K23EY029246, R01AG060942 and an unrestricted grant from Research to Prevent Blindness.

University of Washington Health Sciences/UW Medicine

Related Ophthalmology Articles from Brightsurf:

Practice patterns, responsiveness to common ocular complaints among ophthalmology centers during COVID-19
Practice patterns for common ocular complaints during the initial stage of the COVID-19 pandemic are reported in this observational study among comprehensive U.S. ophthalmology practices.

Detection of COVID-19 viral material on environmental surfaces of an ophthalmology exam room
This study investigates the presence of SARS-CoV-2 on the environmental surfaces of an ophthalmology examination room after visits by patients who were asymptomatic and passed COVID-19 triage.

Ophthalmology after COVID-19
Changes in ophthalmology practices after the COVID-19 pandemic are discussed in this article.

Perspectives on COVID-19 control measures for ophthalmology clinics
This article describes treatment initiatives being undertaken for novel coronavirus 2019 at an ophthalmology center in Singapore.

Low risk of coronavirus spreading through tears
Study published today in the journal of the American Academy of Ophthalmology found no virus in tears of COVID-19 infected patients.

Low-cost, portable system takes OCT beyond ophthalmology
Researchers have developed a way to perform optical coherence tomography (OCT) in hard-to-reach areas of the body such as joints.

Relying on Dr. Google to diagnose eye problems may be dangerous to your health
A study examining the diagnoses generated by WebMD Symptom Checker showed the online tool was correct only 26 percent of the time.

American Academy of Ophthalmology reiterates long-standing guidance on LASIK education
The American Academy of Ophthalmology, the world's leading professional association of eye physicians and surgeons, today reiterated its long-standing guidance for patients considering LASIK vision correction surgery.

Burnout, depression can affect ophthalmology residents, study finds
A Brown University undergraduate led a JAMA Ophthalmology study showing that many ophthalmology residents face burnout and are often unable to participate in wellness initiatives, which has adverse consequences for both residents and patients.

Malcolm Gladwell published in the Journal of the American Academy of Ophthalmology
In his best sellers 'The Tipping Point,' 'Blink' and 'Outliers,' Malcolm Gladwell writes about the unexpected implications of scientific research, urging readers to think different.

Read More: Ophthalmology News and Ophthalmology Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.