EUVIMEDScopri EUVIMED
ValidierungsstudieOpen access

American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study

Christabelle Pabalan, Alvin Stewart, Gabrielle Fisher, Andrew Fisher

Journal of Medical Internet Research · 2026-11-05 · 28 · e90724-e90724

DOI: 10.2196/90724 ↗

At a glance

Validierungsstudie in Journal of Medical Internet Research, published online on 2026-11-05. Written by Christabelle Pabalan and 3 co-authors. Institution: Qventus Inc, Mountain View, CA, United States. Major MEDLINE topics: Machine Learning, Anesthesiologists. 39 references. The full text is freely available in PubMed Central.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

The American Society of Anesthesiologists Physical Status (ASA-PS) classification system fundamentally shapes perioperative care delivery but suffers from poor interrater reliability (0.4‐0.6). Machine learning (ML) models and large language models (LLMs) offer complementary remedies: ML processes structured data consistently but cannot interpret the nuance of clinical narratives, while LLMs parse narrative text and generate explanations but adhere inconsistently to diagnostic criteria.

Objective

This study aimed to develop and evaluate a parallel ML-LLM ensemble that combines the complementary strengths of both approaches for automated ASA-PS classification.

Methods

We retrospectively analyzed 3000 adult surgical encounters from the University of Arkansas for Medical Sciences (UAMS) between August 2024 and May 2025. Cases were randomly allocated to training (n=2000), validation (N=500), and held-out test sets (N=500). The validation set was used exclusively for architecture comparison and hyperparameter tuning, and the held-out test set was reserved for final model performance validation. We developed multiple architectures, including traditional ML models (Extreme Gradient Boosting [XGBoost], Light Gradient Boosting Machine [LightGBM], ExtraTrees), an LLM-only baseline (Generative Pretrained Transformer-4o [GPT-4o]), and hybrid approaches. The parallel ensemble processed structured data through XGBoost and unstructured clinical notes through GPT-4o independently, with outputs combined via weighted averaging. Model performance was evaluated using macro- F 1 -score, exact match accuracy, and ±1-class accuracy.

Results

A theoretical performance ceiling of macro- F 1 -score=0.59 was determined through evaluation by an expert panel of 2 board-certified anesthesiologists (AF and GF) who independently rated 50 patient charts for ASA-PS scores. On the true held-out test set (N=500), the parallel ensemble ( α =0.30) achieved a macro- F 1 -score of 0.530 (95% CI 0.350‐0.600), exact match accuracy of 0.630 (95% CI 0.580‐0.670), and ±1-class accuracy of 0.980 (95% CI 0.968‐0.992), exceeding the frozen LLM-only baseline evaluated on the same set (macro- F 1 -score 0.493, exact match accuracy 0.621,±1-class accuracy 0.981). On the validation set used for architecture selection, the ensemble outperformed structured-only ML (XGBoost macro- F 1 -score=0.333, P

Subjects

MeSH terms (MEDLINE)

Machine LearningAnesthesiologistsLarge Language ModelsRetrospective StudiesHumansBoosting Machine Learning AlgorithmsSocieties, MedicalPredictive Learning Models

Author keywords

ASA physical status classificationASA-PSAmerican Society of Anesthesiologists Physical Status classificationlarge language modelsmachine learningperioperative careperioperative risk stratification

Authors and institutions

  • Qventus Inc, Mountain View, CA, United States.
  • Department of Anesthesiology, University of Arkansas Medical Center, Little Rock, AR, United States.
  • Department Anesthesiology and Perioperative Medicine, Medical University of South Carolina, 135 Rutledge Ave, Charleston, SC, 29425, United States, +1 (843) 792-2322.

Cite

APA

Pabalan, C., Stewart, A., Fisher, G., & Fisher, A. (2026). American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study. Journal of Medical Internet Research, 28, e90724-e90724. https://doi.org/10.2196/90724

Vancouver

Pabalan C, Stewart A, Fisher G, Fisher A. American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study. Journal of Medical Internet Research. 2026;28:e90724-e90724. doi:10.2196/90724

BibTeX
@article{Pabalan2026,
  author = {Pabalan, Christabelle and Stewart, Alvin and Fisher, Gabrielle and Fisher, Andrew},
  title = {American Society of Anesthesiologists Physical Status Classification: Hybrid Machine Learning–Large Language Model Ensemble Retrospective Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e90724--e90724},
  doi = {10.2196/90724}
}
Start for free →