Widely used health care prediction algorithm found to be biased against blacks

October 24, 2019

From predicting who will be a repeat offender to who's the best candidate for a job, computer algorithms are now making complex decisions in lieu of humans. But increasingly, many of these algorithms are being found to replicate the same racial, socioeconomic or gender-based biases they were built to overcome.

This racial bias extends to software widely used in the health care industry, potentially affecting access to care for millions of Americans, according to a new study by researchers at the University of California, Berkeley, the University of Chicago Booth School of Business and Partners HealthCare in Boston.

The new study, published Oct. 25 in the journal Science, found that a type of software program that determines who gets access to high-risk heath care management programs routinely lets healthier whites into the programs ahead of blacks who are less healthy. Fixing this bias in the algorithm could more than double the number of black patients automatically admitted to these programs, the study revealed.

"We found that a category of algorithms that influences health care decisions for over a hundred million Americans shows significant racial bias," said Sendhil Mullainathan, the Roman Family University Professor of Computation and Behavioral Science at Chicago Booth and senior author of the study.

"The algorithms encode racial bias by using health care costs to determine patient 'risk,' or who was mostly likely to benefit from care management programs," said Ziad Obermeyer, acting associate professor of health policy and management at UC Berkeley and lead author of the paper.

"Because of the structural inequalities in our health care system, blacks at a given level of health end up generating lower costs than whites," Obermeyer said. "As a result, black patients were much sicker at a given level of the algorithm's predicted risk."

By tweaking the algorithm to use other variables to predict patient risk, such as costs that could be avoided by preventative care, the researchers were able to correct much of the bias that was initially built into the algorithm.

"Algorithms by themselves are neither good nor bad," Mullainathan said. "It is merely a question of taking care in how they are built. In this case, the problem is eminently fixable -- and at least one manufacturer appears to be working on a fix. We would encourage others to do so."

More generally, said Obermeyer, incorporating routine audits into algorithm developers' workflows would help. "For algorithms, just as for medicine," he said, "we'd prefer to prevent problems, instead of curing them."

Digging up the roots of algorithmic bias

Uncovering algorithmic bias -- be it in the criminal justice system, in hiring decisions or in health care -- is often hindered by the fact that many of the prediction algorithms in use today are designed by private companies and proprietary, making it difficult for data scientists and researchers to analyze them.

To tackle this problem, Mullainathan and Obermeyer partnered with researchers at an academic hospital that was using a risk-based algorithm to determine which patients were getting preferential access to a high-risk care management program. Programs like this are designed to improve care for patients with complex medical needs by providing them with additional attention and resources.

For 43,539 white patients and 6,079 black patients enrolled in the hospital, the researchers obtained the algorithm-predicted risk score and compared it to more direct measures of a patient's health, including number of chronic illnesses and other biomarkers.

They found that, for a given risk score, blacks had significantly poorer health than their white counterparts.

"Instead of being trained to find the sickest, in a physiological sense, [these algorithms] ended up being trained to find the sickest in the sense of those whom we spend the most money on," Mullainathan said. "And there are systemic racial differences in health care in who we spend money on."

Patients whose risk scores landed in the top 97% were automatically identified for enrollment in the care management program. By correcting for the health disparities between blacks and whites, the researchers found that the percentage of black people in the automatic enrollee group jumped from 18% to 47%.

But there is room for hope, Obermeyer said. Training the algorithm to determine risk based on other measurable variables, such as avoidable cost, or the number of chronic conditions that needed treatment in a year, significantly reduced the racial bias.

And when alerted to the bias built into its algorithm, the software manufacturer was very motivated to address the issue, Obermeyer said.

"Algorithms can do terrible things, or algorithms can do wonderful things. Which one of those things they do is basically up to us," Obermeyer said. "We make so many choices when we train an algorithm that feel technical and small. But these choices make the difference between an algorithm that's good or bad, biased or unbiased. So, it's often very understandable when we end up with algorithms that don't do what we want them to do, because those choices are hard."
Co-authors of the study include Brian Powers of Brigham and Women's Hospital in Boston and Christine Vogeli of Massachusetts General Hospital.

This study was funded, in part, by a grant from the National Institute for Health Care Management Foundation.

University of California - Berkeley

Related Health Care Articles from Brightsurf:

Study evaluates new World Health Organization Labor Care Guide for maternity care providers
The World Health Organization developed the new Labor Care Guide to support clinicians in providing good quality, women-centered care during labor and childbirth.

Six ways primary care "medical homes" are lowering health care spending
New analysis of 394 U.S. primary care practices identifies the aspects of care delivery that are associated with lower health care spending and lower utilization of emergency care and hospital admissions.

Modifiable health risks linked to more than $730 billion in US health care costs
Modifiable health risks, such as obesity, high blood pressure, and smoking, were linked to over $730 billion in health care spending in the US in 2016, according to a study published in The Lancet Public Health.

Spending on primary care vs. other US health care expenditures
National health care survey data were used to assess the amount of money spent on primary care relative to other areas of health care spending in the US from 2002 to 2016.

MU Health Care neurologist publishes guidance related to COVID-19 and stroke care
A University of Missouri Health Care neurologist has published more than 40 new recommendations for evaluating and treating stroke patients based on international research examining the link between stroke and novel coronavirus (COVID-19).

Large federal program aimed at providing better health care underfunds primary care
Despite a mandate to help patients make better-informed health care decisions, a ten-year research program established under the Affordable Care Act has funded a relatively small number of studies that examine primary care, the setting where the majority of patients in the US receive treatment.

International medical graduates care for Medicare patients with greater health care needs
A study by a Massachusetts General Hospital research team indicates that internal medicine physicians who are graduates of medical schools outside the US care for Medicare patients with more complex medical needs than those cared for by graduates of American medical schools.

The Lancet Global Health: Improved access to care not sufficient to improve health, as epidemic of poor quality care revealed
Of the 8.6 million deaths from conditions treatable by health care, poor-quality care is responsible for an estimated 5 million deaths per year -- more than deaths due to insufficient access to care (3.6 million) .

Under Affordable Care Act, Americans have had more preventive care for heart health
By reducing out-of-pocket costs for preventive treatment, the Affordable Care Act appears to have encouraged more people to have health screenings related to their cardiovascular health.

High-deductible health care plans curb both cost and usage, including preventive care
A team of researchers based at IUPUI has conducted the first systematic review of studies examining the relationship between high-deductible health care plans and the use of health care services.

Read More: Health Care News and Health Care Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.