Nav: Home

Doc versus machine

October 10, 2016

Increasingly powerful computers using ever-more sophisticated programs are challenging human supremacy in areas as diverse as playing chess and making emotionally compelling music. But can digital diagnosticians match, or even outperform, human physicians?

The answer, according to a new study led by researchers at Harvard Medical School, is "not quite."

The findings, published Oct. 10 in JAMA Internal Medicine, show that physicians' performance is vastly superior and that doctors make a correct diagnosis more than twice as often as 23 commonly used symptom-checker apps. The analysis is believed to provide the first direct comparison between human-made and computer-based diagnoses.

Diagnostic errors stem from failure to recognize a disease or to do so in a timely manner. Physicians make such errors roughly 10 to 15 percent of the time, researchers say.

Over the last two decades, computer-based checklists and other fail-safe digital apps have been increasingly used to reduce medication errors or streamline infection-prevention protocols. Lately, experts have wondered whether computers might also help improve clinical diagnoses and reduce diagnostic errors. Each year, hundreds of millions of people use Internet programs or apps to check their symptoms or to self-diagnose. Yet how these computerized symptom-checkers fare against physicians has not been well studied.

In the study, 234 internal medicine physicians were asked to evaluate 45 clinical cases, involving both common and uncommon conditions with varying degrees of severity. For each scenario, physicians had to identify the most likely diagnosis along with two additional possible diagnoses. Each clinical vignette was solved by at least 20 physicians.

The physicians outperformed the symptom-checker apps, listing the correct diagnosis first 72 percent of the time, compared with 34 percent of the time for the digital platforms. Eighty-four percent of clinicians listed the correct diagnosis in the top three possibilities, compared with 51 percent for the digital symptom-checkers.

The difference between physician and computer performance was most dramatic in more severe and less common conditions. It was smaller for less acute and more common illnesses.

"While the computer programs were clearly inferior to physicians in terms of diagnostic accuracy, it will be critical to study future generations of computer programs that may be more accurate," said senior investigator Ateev Mehrotra, an associate professor of health care policy at HMS.

Despite outperforming the machines, physicians still made errors in about 15 percent of cases. Researchers say developing computer-based algorithms to be used in conjunction with human decision-making may help further reduce diagnostic errors.

"Clinical diagnosis is currently as much art as it is science, but there is great promise for technology to help augment clinical diagnoses," Mehrotra said. "That is the true value proposition of these tools."
-end-
Co-investigators included Hannah Semigran, former research assistant in the HMS Department of Health Care Policy; David Levine, HMS research fellow in medicine at Brigham and Women's Hospital; and Shantanu Nundy, an employee of the Human Diagnosis Project, the creators of Human Dx.

Harvard Medical School has more than 9,500 full-time faculty working in 10 academic departments located at the School's Boston campus or in hospital-based clinical departments at 15 Harvard-affiliated teaching hospitals and research institutes: Beth Israel Deaconess Medical Center, Boston Children's Hospital, Brigham and Women's Hospital, Cambridge Health Alliance, Dana-Farber Cancer Institute, Harvard Pilgrim Health Care Institute, Hebrew SeniorLife, Joslin Diabetes Center, Judge Baker Children's Center, Massachusetts Eye and Ear/Schepens Eye Research Institute, Massachusetts General Hospital, McLean Hospital, Mount Auburn Hospital, Spaulding Rehabilitation Network and VA Boston Healthcare System.

Harvard Medical School

Related Physicians Articles:

Types and distribution of payments from industry to physicians
In 2015, nearly half of physicians were reported to have received a total of $2.4 billion in industry-related payments, primarily involving general payments (including consulting fees and food and beverage), with a higher likelihood and value of payments to physicians in surgical than primary care specialties and to male than female physicians, according to a study published by JAMA in a theme issue on conflict of interest.
Is higher health care spending by physicians associated with better outcomes?
Higher health care utilization spending by physicians was not associated with better outcomes for hospitalized Medicare beneficiaries in a new article published online by JAMA Internal Medicine.
Time for physicians to prepare for impending appropriate use mandate
Within a year, the Centers for Medicare & Medicaid Services (CMS) will implement a provision in the Protecting Access to Medicare Act that requires physicians to consult appropriate use criteria using CMS-approved computer-based clinical decision support mechanisms when ordering advanced imaging procedures.
Four Loyola physicians named to Who's Who in Hispanic Chicago
Four Loyola Medicine physicians have been named to Negocios Now's 2016 'Who's Who in Hispanic Chicago.' Loyola has more physicians on the list than any other medical center.
Physicians are more likely to use hospice and intensive care at end of life
New research suggests that US physicians are more likely to use hospice and intensive or critical care units in the last months of life than non-physicians.
Professional burnout associated with physicians limiting practice
At a time when the nation is facing projected physician shortages, a Mayo Clinic study shows an association between burnout and declining professional satisfaction with physicians reducing the number of hours they devote to clinical practice.
By asking, 'what's the worst part of this?' physicians can ease suffering
When patients suffer, doctors tend to want to fix things and if they cannot many doctors then withdraw emotionally.
Physicians in training at high risk for depression
New research from Brigham and Women's Hospital finds that 28.8 percent of trainees screen positive for depression during their residency.
Physicians and burnout: It's getting worse
Burnout among US physicians is getting worse. An update from a three-year study evaluating burnout and work-life balance shows that American physicians are worse off today than they were three years earlier.
American College of Physicians urges physicians to oppose mass deportation
The American College of Physicians (ACP) today called on physicians, individually and collectively, to speak out against proposals to deport the 12 million US residents who lack documentation of legal residency status, citing the adverse impact that mass deportation would have on individuals and the health of the public.

Related Physicians Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Jumpstarting Creativity
Our greatest breakthroughs and triumphs have one thing in common: creativity. But how do you ignite it? And how do you rekindle it? This hour, TED speakers explore ideas on jumpstarting creativity. Guests include economist Tim Harford, producer Helen Marriage, artificial intelligence researcher Steve Engels, and behavioral scientist Marily Oppezzo.
Now Playing: Science for the People

#524 The Human Network
What does a network of humans look like and how does it work? How does information spread? How do decisions and opinions spread? What gets distorted as it moves through the network and why? This week we dig into the ins and outs of human networks with Matthew Jackson, Professor of Economics at Stanford University and author of the book "The Human Network: How Your Social Position Determines Your Power, Beliefs, and Behaviours".