Calibrating Machine Learning Approaches for Probability Estimation Without Calibration Data
Statistics in Medicine · 2026
DOI: 10.1002/sim.70751 ↗Kurz gefasst
Fachartikel in der Zeitschrift Statistics in Medicine, 2026. Verfasst von Eleonora Di Carluccio und 4 weiteren Autor:innen. Beteiligte Einrichtung: Cardio-CARE, Medizincampus Davos, Davos, Switzerland. Hauptthemen laut MEDLINE: Machine Learning, Models, Statistical. Die Arbeit nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Statistical prediction models for binary outcomes are becoming increasingly popular. One significant challenge is calibrating these models to suit the characteristics of a target population that is structurally different from the original population. Calibration is especially challenging when there is no training data available from the target population. To address this problem, we propose a novel calibration method, SimCal, which uses synthetic data generated from the model development data in conjunction with marginal statistics from the calibration cohort. We show that expert judgment modeling (EJM) may be used for calibration if cross‐sectional data from the target population are available comprising expert judgments about the potential outcome and the covariates. We describe three alternative calibration approaches when calibration data are lacking: similarity‐binning averaging (SBA), adaptive calibration of predictions (ACP), and Elkan calibration. In a simulation study, we compare SBA, ACP, Elkan calibration, and SimCal. R code for applying these methods is provided from the re‐analysis of data on coronary artery disease. We illustrate all 5 calibration approaches with a real data set for predicting functional outcome after stroke and all approaches but EJM in the re‐analysis of the Cleveland Clinic data. None of the approaches performed convincingly well in all situations. SimCal performed well when model parameters were correctly specified. EJM failed on the stroke data. Further research is urgently required for calibration in the absence of calibration data.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Cardio-CARE, Medizincampus Davos, Davos, Switzerland.
- Department of Cardiology, University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
- Center for Population Health Innovation (POINT), University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
- Partner Site Hamburg/Kiel/Lübeck, German Center for Cardiovascular Research (DZHK), Hamburg, Germany.
- BDH-Klinik Elzach, Elzach, Germany.
- Institute for Medical Informatics, Biometry and Epidemiology, University Hospital, University Duisburg-Essen, Essen, Germany.
ORCID-Profile: Eleonora Di Carluccio, Georgios Koliopanos, Andreas Ziegler
Zitieren
APA
Carluccio, E. D., Koliopanos, G., Ojeda, F. M., Weimar, C., & Ziegler, A. (2026). Calibrating Machine Learning Approaches for Probability Estimation Without Calibration Data. Statistics in Medicine. https://doi.org/10.1002/sim.70751
Vancouver
Carluccio ED, Koliopanos G, Ojeda FM, Weimar C, Ziegler A. Calibrating Machine Learning Approaches for Probability Estimation Without Calibration Data. Statistics in Medicine. 2026. doi:10.1002/sim.70751
BibTeX
@article{Carluccio2026,
author = {Carluccio, Eleonora Di and Koliopanos, Georgios and Ojeda, Francisco M and Weimar, Christian and Ziegler, Andreas},
title = {Calibrating Machine Learning Approaches for Probability Estimation Without Calibration Data},
journal = {Statistics in Medicine},
year = {2026},
doi = {10.1002/sim.70751}
}
EUVIMED