Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study
Korean Journal of Anesthesiology · 2026
DOI: 10.4097/kja.25646 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Korean Journal of Anesthesiology, 2026. Verfasst von Jipyeong Lee und 5 weiteren Autor:innen. Beteiligte Einrichtung: Healthcare AI Research Institute, Seoul National University Hospital, Seoul, Korea. Hauptthemen laut MEDLINE: Large Language Models, Machine Learning, Postoperative Complications. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Korea Health Industry Development Institute, Ministry of Health and Welfare, NAVER Corporation.
Abstract (Originalsprache)
Background
Conventional machine learning (ML) models for predicting surgical outcomes have limitations in generalizability. We explored large language models (LLMs) as scalable alternatives to conventional ML models in predicting postoperative outcomes, including in-hospital 30-day mortality, intensive care unit (ICU) admission, and acute kidney injury (AKI).
Methods
This study utilized the Informative Surgical Patient for Innovative Research Environment (INSPIRE) dataset (n = 80 985) from South Korea for model development and internal validation, and the Medical Informatics Operating Room Vitals and Events Repository (MOVER) dataset (n = 6265) from the United States for external validation. The study compared three different LLMs—Generative Pre-trained Transformer [GPT]-4o, Llama-3-70B, and OpenBioLLM-70B—against MLs using various prompt engineering approaches. LLMs were evaluated with different model parameter quantizations (4-bit normalized floating point vs. 16-bit brain floating point).
Results
OpenBioLLM-70B was comparable to eXtreme Gradient Boosting (XGBoost) across all tasks (in-hospital 30-day mortality: area under receiver operating characteristic curve [AUROC] 0.782 [95% CI, 0.748–0.813] vs. 0.791 [95% CI, 0.753–0.825]; ICU admission: AUROC 0.595 [95% CI, 0.581–0.609] vs. 0.594 [95% CI, 0.580–0.608]; AKI: AUROC 0.830 [95% CI, 0.802–0.855] vs. 0.823 [95% CI, 0.792–0.851]) during external validation. Open-source LLMs maintained performance with 4-bit quantization, reducing computational requirements by 75%.
Conclusions
The findings support the versatility and efficiency of LLMs for clinical decision support through on-premises compatibility, addressing data privacy. Further validation with diverse datasets is needed to ensure their reliability and applicability across different perioperative settings.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Healthcare AI Research Institute, Seoul National University Hospital, Seoul, Korea.
- Department of Interdisciplinary Program in Medical Informatics, Seoul National University, Seoul, Korea.
- Monash School of Medicine, Monash University, Victoria, Australia.
- Department of Anesthesiology and Pain Medicine, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Korea.
- Healthcare AI Research Institute, Seoul National University Hospital, Seoul, Korea. hhoon@snu.ac.kr.
- Department of Medicine, Seoul National University College of Medicine, Seoul, Korea. hhoon@snu.ac.kr.
Förderung
- Korea Health Industry Development Institute
- Ministry of Health and Welfare
- NAVER Corporation
Zitieren
APA
Lee, J., Kim, H., Kim, L., Lim, L., Lee, H. C., & Lee, H. (2026). Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study. Korean Journal of Anesthesiology. https://doi.org/10.4097/kja.25646
Vancouver
Lee J, Kim H, Kim L, Lim L, Lee HC, Lee H. Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study. Korean Journal of Anesthesiology. 2026. doi:10.4097/kja.25646
BibTeX
@article{Lee2026,
author = {Lee, Jipyeong and Kim, Hyeonsik and Kim, Luke and Lim, Leerang and Lee, Hyung-Chul and Lee, Hyeonhoon},
title = {Comparison of large language models and conventional machine learning in postoperative outcome prediction: a retrospective, multi-national development and validation study},
journal = {Korean Journal of Anesthesiology},
year = {2026},
doi = {10.4097/kja.25646}
}
EUVIMED