Performances of five large language models in clinical decision-making for internal medicine: A comparative study
DIGITAL HEALTH · 2026-02 · 12
DOI: 10.1177/20552076261433073 ↗Kurz gefasst
Fachartikel in der Zeitschrift DIGITAL HEALTH, online erschienen am 25.03.2026. Verfasst von Dan Wu und 2 weiteren Autor:innen. Beteiligte Einrichtung: Integrated Traditional Chinese and Western Medicine Clinical Center, The Affiliated Guangdong Second Provincial General Hospital of Jinan University,…. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Background
This study evaluated the performance of five Large Language Models based on actual cases to provide guidance for selecting appropriate models for clinical decision-making.
Objective
This study aimed to assess the performance of large language models (LLMs) in clinical decision-making for internal medicine and to provide evidence-based guidance for model selection in clinical practice.
Methods
We conducted a retrospective cross-sectional study with 405 cases across nine subspecialties: cardiovascular, respiratory, gastroenterology, nephrology, rheumatology, endocrinology, neurology, hematology, and infectious diseases. Two senior clinicians evaluated outputs on five dimensions: diagnosis, diagnostic criteria, differential diagnosis, examinations, and treatment. Statistical analyses were performed via the Kruskal‒Wallis tests and Pairwise comparisons were performed by Dunn's test with p-value adjusted by BH procedure.
Results
Overall, significant performance differences were observed among models ( p = 0.001). All models performed worst in respiratory ( p < 0.05). Gemini significantly outperformed others in differential diagnosis—the weakest area across all models ( p < 0.05). Claude scored significantly lower than other LLMs in Card and Heme ( p < 0.05). Subgroup analysis indicated that the most pronounced performance disparities were observed in the Card ( p < 0.05).
Conclusion
GPT, O1, and Gemini demonstrated superior performance in clinical decision-making for internal medicine among all LLMs, whereas Claude showed the poorest performance. All LLMs demonstrated deficiencies in differential diagnosis and poor management for respiratory diseases. The complexity of subspecialty might be a performance differentiator for LLMs and O1 might have potential suitability for complex subspecialties like cardiology.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Integrated Traditional Chinese and Western Medicine Clinical Center, The Affiliated Guangdong Second Provincial General Hospital of Jinan University, Guangzhou, Guangdong, China.
- Rehabilitation Center, The First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, Guangdong, China.
- Guangzhou Yuansheng Ruihang Medical Information Technology Co., Guangzhou, Guangdong, China.
- Huazhou Community Health Service Center of Haizhu District of Guangzhou, Guangzhou, Guangdong, China.
ORCID-Profile: Dan Wu, Jingjing Lu, Danghan Xu
Zitieren
APA
Wu, D., Lu, J., & Xu, D. (2026). Performances of five large language models in clinical decision-making for internal medicine: A comparative study. DIGITAL HEALTH, 12. https://doi.org/10.1177/20552076261433073
Vancouver
Wu D, Lu J, Xu D. Performances of five large language models in clinical decision-making for internal medicine: A comparative study. DIGITAL HEALTH. 2026;12. doi:10.1177/20552076261433073
BibTeX
@article{Wu2026,
author = {Wu, Dan and Lu, Jingjing and Xu, Danghan},
title = {Performances of five large language models in clinical decision-making for internal medicine: A comparative study},
journal = {DIGITAL HEALTH},
year = {2026},
volume = {12},
doi = {10.1177/20552076261433073}
}
EUVIMED