Calibration plots for multistate risk prediction models
Statistics in Medicine · 30.06.2024 · 43 (14) · 2830-2852
DOI: 10.1002/sim.10094 ↗Kurz gefasst
Fachartikel in der Zeitschrift Statistics in Medicine, online erschienen am 08.05.2024. Verfasst von Alexander Pate und 11 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Health Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Healt…. Hauptthemen laut MEDLINE: Diabetes Mellitus, Type 2, Models, Statistical, Computer Simulation. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 103 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Medical Research Council.
Abstract (Originalsprache)
Introduction
There is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation.
Methods
We studied pseudo‐values based on the Aalen‐Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR‐IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR‐IPCW). The MLR‐IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed and evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records.
Results
The pseudo‐value, BLR‐IPCW, and MLR‐IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low‐density regions of predicted transition probabilities.
Conclusions
We recommend implementing either the pseudo‐value or BLR‐IPCW approaches to produce a calibration curve, combined with the MLR‐IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the “calibmsm” R package available on CRAN.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Centre for Health Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.
- NIHR Manchester Biomedical Research Centre, University of Manchester, Manchester, UK.
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK.
- Centre for Biostatistics, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK.
- Keele Cardiovascular Research Group, Keele University, Stoke-on-Trent, UK.
ORCID-Profile: Alexander Pate, Matthew Sperrin, Glen P Martin
Förderung
- Medical Research Council
Zitieren
APA
Pate, A., Sperrin, M., Riley, R. D., Peek, N., Van Staa, T., Sergeant, J. C., Mamas, M. A., Lip, G. Y. H., O'Flaherty, M., Barrowman, M., Buchan, I., & Martin, G. P. (2024). Calibration plots for multistate risk prediction models. Statistics in Medicine, 43(14), 2830-2852. https://doi.org/10.1002/sim.10094
Vancouver
Pate A, Sperrin M, Riley RD, Peek N, Van Staa T, Sergeant JC, et al. Calibration plots for multistate risk prediction models. Statistics in Medicine. 2024;43(14):2830-2852. doi:10.1002/sim.10094
BibTeX
@article{Pate2024,
author = {Pate, Alexander and Sperrin, Matthew and Riley, Richard D and Peek, Niels and Van Staa, Tjeerd and Sergeant, Jamie C and Mamas, Mamas A and Lip, Gregory Y H and O'Flaherty, Martin and Barrowman, Michael and Buchan, Iain and Martin, Glen P},
title = {Calibration plots for multistate risk prediction models},
journal = {Statistics in Medicine},
year = {2024},
volume = {43},
number = {14},
pages = {2830--2852},
doi = {10.1002/sim.10094}
}
EUVIMED