American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study
Journal of Medical Internet Research · 05.11.2026 · 28 · e90724-e90724
DOI: 10.2196/90724 ↗Kurz gefasst
Validierungsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 05.11.2026. Verfasst von Christabelle Pabalan und 3 weiteren Autor:innen. Beteiligte Einrichtung: Qventus Inc, Mountain View, CA, United States. Hauptthemen laut MEDLINE: Machine Learning, Anesthesiologists. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
The American Society of Anesthesiologists Physical Status (ASA-PS) classification system fundamentally shapes perioperative care delivery but suffers from poor interrater reliability (0.4‐0.6). Machine learning (ML) models and large language models (LLMs) offer complementary remedies: ML processes structured data consistently but cannot interpret the nuance of clinical narratives, while LLMs parse narrative text and generate explanations but adhere inconsistently to diagnostic criteria.
Objective
This study aimed to develop and evaluate a parallel ML-LLM ensemble that combines the complementary strengths of both approaches for automated ASA-PS classification.
Methods
We retrospectively analyzed 3000 adult surgical encounters from the University of Arkansas for Medical Sciences (UAMS) between August 2024 and May 2025. Cases were randomly allocated to training (n=2000), validation (N=500), and held-out test sets (N=500). The validation set was used exclusively for architecture comparison and hyperparameter tuning, and the held-out test set was reserved for final model performance validation. We developed multiple architectures, including traditional ML models (Extreme Gradient Boosting [XGBoost], Light Gradient Boosting Machine [LightGBM], ExtraTrees), an LLM-only baseline (Generative Pretrained Transformer-4o [GPT-4o]), and hybrid approaches. The parallel ensemble processed structured data through XGBoost and unstructured clinical notes through GPT-4o independently, with outputs combined via weighted averaging. Model performance was evaluated using macro- F 1 -score, exact match accuracy, and ±1-class accuracy.
Results
A theoretical performance ceiling of macro- F 1 -score=0.59 was determined through evaluation by an expert panel of 2 board-certified anesthesiologists (AF and GF) who independently rated 50 patient charts for ASA-PS scores. On the true held-out test set (N=500), the parallel ensemble ( α =0.30) achieved a macro- F 1 -score of 0.530 (95% CI 0.350‐0.600), exact match accuracy of 0.630 (95% CI 0.580‐0.670), and ±1-class accuracy of 0.980 (95% CI 0.968‐0.992), exceeding the frozen LLM-only baseline evaluated on the same set (macro- F 1 -score 0.493, exact match accuracy 0.621,±1-class accuracy 0.981). On the validation set used for architecture selection, the ensemble outperformed structured-only ML (XGBoost macro- F 1 -score=0.333, P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Qventus Inc, Mountain View, CA, United States.
- Department of Anesthesiology, University of Arkansas Medical Center, Little Rock, AR, United States.
- Department Anesthesiology and Perioperative Medicine, Medical University of South Carolina, 135 Rutledge Ave, Charleston, SC, 29425, United States, +1 (843) 792-2322.
Zitieren
APA
Pabalan, C., Stewart, A., Fisher, G., & Fisher, A. (2026). American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study. Journal of Medical Internet Research, 28, e90724-e90724. https://doi.org/10.2196/90724
Vancouver
Pabalan C, Stewart A, Fisher G, Fisher A. American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study. Journal of Medical Internet Research. 2026;28:e90724-e90724. doi:10.2196/90724
BibTeX
@article{Pabalan2026,
author = {Pabalan, Christabelle and Stewart, Alvin and Fisher, Gabrielle and Fisher, Andrew},
title = {American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e90724--e90724},
doi = {10.2196/90724}
}
EUVIMED