Tipping the balance: impact of class imbalance correction on the performance of clinical risk prediction models
Journal of the American Medical Informatics Association · 01.10.2026 · 33 (10) · 1943-1956
DOI: 10.1093/jamia/ocag127 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 30.07.2026. Verfasst von Amalie Koch Andersen und 10 weiteren Autor:innen. Beteiligte Einrichtung: Medical Informatics and Image Analysis, Department of Health Science and Technology, Aalborg University, Aalborg, 9260, Denmark. Hauptthemen laut MEDLINE: Machine Learning. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Novo Nordisk Foundation, Danish Diabetes and Endocrine Academy, Danish Data Science Academy.
Abstract (Originalsprache)
Objectives
Machine-learning–based clinical risk prediction models are increasingly used to support decision-making in healthcare. While class-imbalance correction techniques are commonly applied to address rare outcomes, their impact on probabilistic calibration remains insufficiently understood. This study evaluated the effect of widely used resampling strategies on both discrimination and calibration across real-world clinical prediction tasks. Materials and Methods Ten clinical datasets spanning diverse medical domains and including over 600 000 patients were analyzed. Multiple machine-learning model families were evaluated. Models were trained on original data and using 3 1:1 class-imbalance correction strategies (synthetic minority oversampling technique, random undersampling, and random oversampling). Performance was assessed on held-out data using discrimination and calibration metrics.
Results
Resampling had no positive impact on predictive performance. Changes in area under the receiver operating characteristic curve (ROC-AUC) and precision-recall AUC were small and inconsistent (ROC-AUC: −0.002 to −0.01; PR-AUC: −0.10 to −0.03), with no method showing systematic improvement. In contrast, calibration was consistently degraded. Resampled models showed higher Brier scores (increase 0.029–0.080) and marked deviations in calibration intercept and slope, indicating distorted predicted risks despite preserved ranking performance.
Discussion
Across diverse clinical datasets, resampling primarily altered the implicit class prior learned during training, leading to miscalibration when models were evaluated. The consistent dissociation between discrimination and calibration highlights that rank-based metrics alone are insufficient for evaluating clinical utility. Gains from imbalance correction can typically be reproduced by threshold adjustment without distorting predicted probabilities.
Conclusion
Common 1:1 class-imbalance correction techniques do not improve discrimination and may substantially degrade calibration, limiting their suitability for clinical risk prediction where accurate probabilities are essential.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Medical Informatics and Image Analysis, Department of Health Science and Technology, Aalborg University, Aalborg, 9260, Denmark.
- Medical and Translational Science, Novo Nordisk A/S, Søborg, 2860, Denmark.
- Department of Computer Science, Aalborg University, Aalborg, 9220, Denmark.
- Department of Computer Science, Bowling Green State University, Bowling Green, OH, 43403, United States.
- Cardiology, Angiology, Haemostaseology, and Medical Intensive Care, Medical Centre Mannheim, Medical Faculty Mannheim, Heidelberg University, Heidelberg, 69117, Germany.
- European Centre for AngioScience (ECAS), German Centre for Cardiovascular Research (DZHK) partner site Heidelberg/Mannheim, and Centre for Cardiovascular Acute Medicine Mannheim (ZKAM), Medical Centre Mannheim, Medical…
ORCID-Profile: Morten Bøttcher, Simon Winther, Simon Lebech Cichosz
Förderung
- Novo Nordisk Foundation
- Danish Diabetes and Endocrine Academy
- Danish Data Science Academy
Zitieren
APA
Andersen, A. K., Mehdizavareh, H., Khan, A., Becher, T., Britsch, S., Britsch, M., Bøttcher, M., Winther, S., Rohde, P. D., Jensen, M. H., & Cichosz, S. L. (2026). Tipping the balance: impact of class imbalance correction on the performance of clinical risk prediction models. Journal of the American Medical Informatics Association, 33(10), 1943-1956. https://doi.org/10.1093/jamia/ocag127
Vancouver
Andersen AK, Mehdizavareh H, Khan A, Becher T, Britsch S, Britsch M, et al. Tipping the balance: impact of class imbalance correction on the performance of clinical risk prediction models. Journal of the American Medical Informatics Association. 2026;33(10):1943-1956. doi:10.1093/jamia/ocag127
BibTeX
@article{Andersen2026,
author = {Andersen, Amalie Koch and Mehdizavareh, Hadi and Khan, Arijit and Becher, Tobias and Britsch, Simone and Britsch, Markward and Bøttcher, Morten and Winther, Simon and Rohde, Palle Duun and Jensen, Morten Hasselstrøm and Cichosz, Simon Lebech},
title = {Tipping the balance: impact of class imbalance correction on the performance of clinical risk prediction models},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
volume = {33},
number = {10},
pages = {1943--1956},
doi = {10.1093/jamia/ocag127}
}
EUVIMED