Subgroup deployment-readiness audit of calibration and clinical utility in perioperative infection prediction
Journal of the American Medical Informatics Association · 30.10.2026
DOI: 10.1093/jamia/ocag184 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 30.10.2026. Verfasst von Hugo Guillen-Ramirez und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Surgery, Fribourg Hospital, University of Fribourg, 1752 Fribourg, Switzerland. Die Arbeit nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Swiss Personalized Health Network.
Abstract (Originalsprache)
Objective
We audited deployment readiness of an end-of-surgery postoperative infection model across clinically relevant subgroups and assessed mitigation of subgroup miscalibration. Materials and Methods We analyzed out-of-fold predictions from 10 719 procedures (2014-2022) with 504 present-on-admission (POA)–coded postoperative infection events across recorded sex, age, and a clinical physiological-reserve proxy. We assessed discrimination, calibration, threshold-specific errors, and decision-curve net benefit, treating t = 0.10 as a reference anchor. We compared no recalibration with common and group-specific cross-fitted isotonic recalibration, evaluated a refit without the Synthetic Minority Oversampling Technique (SMOTE), and performed a temporal-holdout analysis.
Results
Overall area under the receiver operating characteristic curve (AUROC) was 0.876, with similar sex-marginal discrimination in women and men (0.878 vs 0.875). Calibration-in-the-large was −1.02 overall, −0.86 in frail, and −1.34 in prefrail patients; the 6-event nonfrail stratum was not estimable. At t = 0.10, net benefit was positive in frail and negative in prefrail patients in the source cohort, but negative prefrail utility did not persist in the ascertainable-period (2018-2022) and temporal analyses. Common recalibration moved overall calibration-in-the-large from −1.020 to −0.007. Removing SMOTE shifted overall calibration but did not reduce the reserve gradient. Temporal-holdout AUROC was 0.851 with persistent overprediction. Present-on-admission coding was effectively absent before 2018; prevalence-dependent results were therefore additionally reported for the ascertainable 2018-2022 period, and ascertainment differed by physiological reserve.
Discussion
Discrimination-only or sex-marginal evaluation would have missed deployment-relevant subgroup heterogeneity, while endpoint ascertainment affected prevalence-dependent interpretation.
Conclusion
Joint subgroup evaluation can identify deployment-relevant limitations missed by aggregate metrics. Prospective validation with comparable outcome ascertainment remains necessary before clinical deployment.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Surgery, Fribourg Hospital, University of Fribourg, 1752 Fribourg, Switzerland.
- Department of Visceral Surgery and Medicine, Inselspital, Bern University Hospital, University of Bern, 3010 Bern, Switzerland.
- Department of Digital Medicine, University of Bern, 3008 Bern, Switzerland.
- Multidisciplinary Center for Infectious Diseases, University of Bern, 3012 Bern, Switzerland.
- Medical Directorate, Medizincontrolling, Inselspital, University Hospital Bern, Insel Gruppe, 3010 Bern, Switzerland.
- Department of Clinical Chemistry, Inselspital, Bern University Hospital, University of Bern, 3010 Bern, Switzerland.
ORCID-Profile: Hugo Guillen-Ramirez, Katharina Lucia Lucas, Yves Max Wintsch, Tobias Ueli Blatter, Karen Triep, Olga Endrich, Guido Beldi
Förderung
- Swiss Personalized Health Network
Zitieren
APA
Guillen-Ramirez, H., Lucas, K. L., Wintsch, Y. M., Blatter, T. U., Triep, K., Endrich, O., & Beldi, G. (2026). Subgroup deployment-readiness audit of calibration and clinical utility in perioperative infection prediction. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocag184
Vancouver
Guillen-Ramirez H, Lucas KL, Wintsch YM, Blatter TU, Triep K, Endrich O, et al. Subgroup deployment-readiness audit of calibration and clinical utility in perioperative infection prediction. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocag184
BibTeX
@article{GuillenRamirez2026,
author = {Guillen-Ramirez, Hugo and Lucas, Katharina Lucia and Wintsch, Yves Max and Blatter, Tobias Ueli and Triep, Karen and Endrich, Olga and Beldi, Guido},
title = {Subgroup deployment-readiness audit of calibration and clinical utility in perioperative infection prediction},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
doi = {10.1093/jamia/ocag184}
}
EUVIMED