BEER-SHEVA, Israel, October 20, 2026 – In an era where patients and clinicians increasingly turn to artificial intelligence for medical advice, a critical question emerges: does AI know when not to test or treat? A new joint study from Ben-Gurion University of the Negev (BGU) and Clalit Health Services reveals that leading Large Language Models (LLMs) successfully identify when medical interventions are redundant in the vast majority of cases, stumbling primarily in gray areas where medical consensus itself is divided.
The peer-reviewed study was published today in Annals of Internal Medicine , the flagship journal of the American College of Physicians ( https://www.acpjournals.org/doi/10.7326/ANNALS-26-01667 ).
The "Just in Case" Dilemma in Modern Medicine
Patients frequently request extra tests "just to be safe." Yet in clinical practice, more testing is not always better medicine. Unnecessary tests often uncover incidental, harmless findings that trigger cascades of anxiety, specialist referrals, invasive biopsies, and surgeries that were never needed in the first place—wasting precious healthcare resources and risking patient harm. Conversely, an overly conservative diagnostic system risks missing crucial interventions when real danger lurks. A clinically safe AI tool must skillfully navigate between these two failure modes.
The study was led by Dr. Dean Ariel , a family physician and researcher at Clalit Health Services (Tel Aviv District) and medical AI researcher and Dr. Guy Laban of BGU's Department of Industrial Engineering and Management, in collaboration with Dr. Stav Endelman of Rabin Medical Center-Beilinson Hospital.
Testing AI Against Real-World Clinical Judgment
The researchers evaluated six leading LLMs developed by Google, Anthropic, and OpenAI across hundreds of clinical scenarios, analyzing thousands of total model responses. The scenarios were constructed around recommendations from the international Choosing Wisely campaign—a global medical initiative founded in 2012 by the American Board of Internal Medicine (ABIM) Foundation and active in more than 30 countries to eliminate low-value care.
Each recommendation paired cases where an intervention was redundant with closely matched controls where the same intervention was medically necessary:
The findings:
Echoing Human Disagreements
The vast majority of AI over-testing errors clustered around two notoriously ambiguous clinical practices:
"One of the hardest parts of practicing medicine is knowing when to hold back," explains Dr. Dean Ariel of Clalit Health Services. "Patients find comfort in an extra test, but an unnecessary MRI often catches an asymptomatic disc bulge, setting off unnecessary worry and treatment. Language models learned medicine from human clinical literature—including our less disciplined habits. We wanted to see if they also absorbed the ability to exercise restraint. For the most part, they know when to stop, stumbling primarily where we doctors ourselves hesitate."
The Social Challenge of Saying "No"
"People often hear an AI chatbot speak in a single authoritative voice, as if there is a unified 'AI consensus,'" notes Dr. Guy Laban of Ben-Gurion University of the Negev. "Yet in gray areas where experts debate, the disparity between models was striking: on the exact same clinical cases, one model recommended an unnecessary test less than 10% of the time, while another recommended it 80% of the time."
"Furthermore, telling a patient 'you don't need this test' is a delicate social act," Dr. Laban adds. "A human physician must stand firm before an anxious patient, whereas language models tend to go along with the person in front of them. The next critical research question is whether an AI assistant will maintain the correct clinical restraint when an anxious or insistent patient pushes back."
About the Research Team
The study was conducted by:
Annals of Internal Medicine
Data/statistical analysis
Not applicable
Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios
20-Oct-2026