Proposed framework for integrating chatbots into health care

July 24, 2020

While the technology for developing artificial intelligence-powered chatbots has existed for some time, a new viewpoint piece in JAMA lays out the clinical, ethical, and legal aspects that must be considered before applying them in healthcare. And while the emergence of COVID-19 and the social distancing that accompanies it has prompted more health systems to explore and apply automated chatbots, the authors still urge caution and thoughtfulness before proceeding.

"We need to recognize that this is relatively new technology and even for the older systems that were in place, the data are limited," said the viewpoint's lead author, John D. McGreevey III, MD, an associate professor of Medicine in the Perelman School of Medicine at the University of Pennsylvania. "Any efforts also need to realize that much of the data we have comes from research, not widespread clinical implementation. Knowing that, evaluation of these systems must be robust when they enter the clinical space, and those operating them should be nimble enough to adapt quickly to feedback."

McGreevey, joined by C. William Hanson III, MD, chief medical information officer at Penn Medicine, and Ross Koppel, PhD, FACMI, a senior fellow at the Leonard Davis Institute of Healthcare Economics at Penn and professor of Medical Informatics, wrote "Clinical, Legal, and Ethical Aspects of AI-Assisted Conversational Agents." In it, the authors lay out 12 different focus areas that should be considered when planning to implement a chatbot, or, more formally, "conversational agent," in clinical care.

Chatbots are a tool used to communicate with patients via text message or voice. Many chatbots are powered by artificial intelligence (AI). This paper specifically discusses chatbots that use natural language processing, an AI process that seeks to "understand" language used in conversations and draws threads and connections from them to provide meaningful and useful answers.

With health care, those messages, and people's reactions to them, are extremely important and carry tangible consequences.

"We are increasingly in direct communication with our patients through electronic medical records, giving them direct access to their test results, diagnoses and doctors' notes," Hanson said. "Chatbots have the ability to enhance the value of those communications on the one hand, or cause confusion or even harm, on the other."

For instance, how a chatbot handles someone telling it something as serious as "I want to hurt myself" has many different implications.

In the self-harm example, there are several areas of focus laid out by the authors that apply. This touches first and foremost on the "Patient Safety" category: Who monitors the chatbot and how often do they do it? It also touches on "Trust and Transparency": Would this patient actually take a response from a known chatbot seriously? It also, unfortunately, raises questions in the paper's "Legal & Licensing" category: Who is accountable if the chatbot fails in its task. Moreover, a question under the "Scope" category may apply here, too: Is this a task best suited for a chatbot, or is it something that should still be totally human-operated?

Within their viewpoint, the team believes they have laid out key considerations that can inform a framework for decision-making when it comes to implementing chatbots in health care. Their considerations should apply even when rapid implementation is required to respond to events like the spread of COVID-19.

"To what extent should chatbots be extending the capabilities of clinicians, which we'd call augmented intelligence, or replacing them through totally artificial intelligence?" Koppel said. "Likewise, we need to determine the limits of chatbot authority to perform in different clinical scenarios, such as when a patient indicates that they have a cough, should the chatbot only respond by letting a nurse know or digging in further: 'Can you tell me more about your cough?'"

Chatbots have the opportunity to significantly improve health outcomes and lower health systems' operating costs, but evaluation and research will be key to that: both to ensure smooth operation and to keep the trust of both patients and health care workers.

"It's our belief that the work is not done when the conversational agent is deployed," McGreevey said. "These are going to be increasingly impactful technologies that deserve to be monitored not just before they are launched, but continuously throughout the life cycle of their work with patients."

University of Pennsylvania School of Medicine

Related Health Care Articles from Brightsurf:

Study evaluates new World Health Organization Labor Care Guide for maternity care providers
The World Health Organization developed the new Labor Care Guide to support clinicians in providing good quality, women-centered care during labor and childbirth.

Six ways primary care "medical homes" are lowering health care spending
New analysis of 394 U.S. primary care practices identifies the aspects of care delivery that are associated with lower health care spending and lower utilization of emergency care and hospital admissions.

Modifiable health risks linked to more than $730 billion in US health care costs
Modifiable health risks, such as obesity, high blood pressure, and smoking, were linked to over $730 billion in health care spending in the US in 2016, according to a study published in The Lancet Public Health.

Spending on primary care vs. other US health care expenditures
National health care survey data were used to assess the amount of money spent on primary care relative to other areas of health care spending in the US from 2002 to 2016.

MU Health Care neurologist publishes guidance related to COVID-19 and stroke care
A University of Missouri Health Care neurologist has published more than 40 new recommendations for evaluating and treating stroke patients based on international research examining the link between stroke and novel coronavirus (COVID-19).

Large federal program aimed at providing better health care underfunds primary care
Despite a mandate to help patients make better-informed health care decisions, a ten-year research program established under the Affordable Care Act has funded a relatively small number of studies that examine primary care, the setting where the majority of patients in the US receive treatment.

International medical graduates care for Medicare patients with greater health care needs
A study by a Massachusetts General Hospital research team indicates that internal medicine physicians who are graduates of medical schools outside the US care for Medicare patients with more complex medical needs than those cared for by graduates of American medical schools.

The Lancet Global Health: Improved access to care not sufficient to improve health, as epidemic of poor quality care revealed
Of the 8.6 million deaths from conditions treatable by health care, poor-quality care is responsible for an estimated 5 million deaths per year -- more than deaths due to insufficient access to care (3.6 million) .

Under Affordable Care Act, Americans have had more preventive care for heart health
By reducing out-of-pocket costs for preventive treatment, the Affordable Care Act appears to have encouraged more people to have health screenings related to their cardiovascular health.

High-deductible health care plans curb both cost and usage, including preventive care
A team of researchers based at IUPUI has conducted the first systematic review of studies examining the relationship between high-deductible health care plans and the use of health care services.

Read More: Health Care News and Health Care Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to