EUVIMEDEUVIMED entdecken
EvaluationsstudieOpen Access

Augmenting clinical decision-making with large language models: evaluation across general and specialty tasks

Yuechen Tao, Xiushi Lin, He Sun, Wenrui Bao, Xiaobo Zhu, Shiwei Wu, Yixiao Liu, Ziang Li, Jialong Wu, Zhenkun Zhao, Lei Qiu, Qianyang Ni und 7 weitere

Journal of the American Medical Informatics Association · 2026

DOI: 10.1093/jamia/ocag053 ↗

Kurz gefasst

Evaluationsstudie in der Zeitschrift Journal of the American Medical Informatics Association, 2026. Verfasst von Yuechen Tao und 18 weiteren Autor:innen. Beteiligte Einrichtung: CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China. Hauptthemen laut MEDLINE: Large Language Models, Clinical Decision-Making, Decision Support Systems, Clinical. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Beijing Natural Science Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Large language models (LLMs) have shown promise in clinical applications, yet prior studies mainly evaluated their standalone performance on benchmarks or examinations. This study assesses how LLMs support clinicians across specialties, disease contexts, experience levels and decision-making stages.

Methods

We evaluated 3 LLMs (Deepseek-R1, GPT-4o-mini and LLaMA-4) using 2 task types: (1) general-disease tasks spanning specialties and disease incidence levels, and (2) prostate disease scenario covering the clinical workflow, including diagnosis, treatment planning, postoperative rehabilitation and prognosis. Clinicians with different seniority completed tasks independently and repeated them after reviewing LLM-generated responses. Performance was rated by experts using a 5-point Likert scale, and differences were analyzed with the Wilcoxon signed-rank test.

Results

LLM assistance significantly improved clinician performance across general disease tasks (P < .05). In the specialty scenario, junior clinicians’ scores increased by 0.579-0.723 (15.9%-20.8%) across 4 clinical stages, whereas senior clinicians’ improvements ranged from 0.053-0.303 (1.3%-7.5%). With LLM support, junior clinicians surpassed senior clinicians’ unaided performance in postoperative rehabilitation and prognosis stages (P < .05) and achieved comparable performance in diagnosis and treatment stages (P > .05). Performance varied among models, with Deepseek-R1 performing best in diagnostic tasks.

Discussion

These findings suggest that LLM assistance may provide greater benefit to less-experienced clinicians and are particularly effective in downstream decision-making stages, indicating a potential role in mitigating experience-related performance disparities.

Conclusions

LLMs may serve as supportive clinical-decision support tools across diseases, specialties, clinician experience levels, and workflow stages, with observed improvements in decision quality and reduced performance disparities.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsClinical Decision-MakingDecision Support Systems, ClinicalHumansMale

Stichwörter der Autor:innen

Large Language Modelsclinical decision-makingclinician experience levelhealthcare augmentationmedical artificial intelligence

Autor:innen und Institutionen

  • CAS Key Laboratory of Molecular Imaging, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China.
  • School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, 100049, China.
  • Department of Urology, Peking University Third Hospital, Beijing, 100191, China.
  • School of Engineering Medicine and School of Biological Science and Medical Engineering, Beihang University, Beijing, 100191, China.
  • School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China.
  • Faculty of Data Science, City University of Macau, Macau, 999078, China.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Beijing Natural Science Foundation
  • Young Elite Scientists Sponsorship Program

Zitieren

APA

Tao, Y., Lin, X., Sun, H., Bao, W., Zhu, X., Wu, S., Liu, Y., Li, Z., Wu, J., Zhao, Z., Qiu, L., Ni, Q., Zhang, S., Tang, Z., Ying, Z., An, Y., Tian, J., Liu, Z., & Lu, J. (2026). Augmenting clinical decision-making with large language models: evaluation across general and specialty tasks. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocag053

Vancouver

Tao Y, Lin X, Sun H, Bao W, Zhu X, Wu S, et al. Augmenting clinical decision-making with large language models: evaluation across general and specialty tasks. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocag053

BibTeX
@article{Tao2026,
  author = {Tao, Yuechen and Lin, Xiushi and Sun, He and Bao, Wenrui and Zhu, Xiaobo and Wu, Shiwei and Liu, Yixiao and Li, Ziang and Wu, Jialong and Zhao, Zhenkun and Qiu, Lei and Ni, Qianyang and Zhang, Song and Tang, Zhenchao and Ying, Zuobin and An, Yu and Tian, Jie and Liu, Zhenyu and Lu, Jian},
  title = {Augmenting clinical decision-making with large language models: evaluation across general and specialty tasks},
  journal = {Journal of the American Medical Informatics Association},
  year = {2026},
  doi = {10.1093/jamia/ocag053}
}
Kostenlos starten →