EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Calibration plots for multistate risk prediction models

Alexander Pate iD, Matthew Sperrin iD, Richard D Riley, Niels Peek, Tjeerd Van Staa, Jamie C Sergeant, Mamas A Mamas, Gregory Y H Lip, Martin O'Flaherty, Michael Barrowman, Iain Buchan, Glen P Martin iD

Statistics in Medicine · 30.06.2024 · 43 (14) · 2830-2852

DOI: 10.1002/sim.10094 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Statistics in Medicine, online erschienen am 08.05.2024. Verfasst von Alexander Pate und 11 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Health Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Healt…. Hauptthemen laut MEDLINE: Diabetes Mellitus, Type 2, Models, Statistical, Computer Simulation. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 103 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Medical Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

There is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation.

Methods

We studied pseudo‐values based on the Aalen‐Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR‐IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR‐IPCW). The MLR‐IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed and evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records.

Results

The pseudo‐value, BLR‐IPCW, and MLR‐IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low‐density regions of predicted transition probabilities.

Conclusions

We recommend implementing either the pseudo‐value or BLR‐IPCW approaches to produce a calibration curve, combined with the MLR‐IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the “calibmsm” R package available on CRAN.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Diabetes Mellitus, Type 2Models, StatisticalComputer SimulationHumansRisk AssessmentLogistic ModelsCalibrationCardiovascular DiseasesRenal Insufficiency, ChronicProbability

Stichwörter der Autor:innen

calibrationclinical predictionmodel validationmultistate modelrisk prediction

Autor:innen und Institutionen

  • Centre for Health Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.
  • NIHR Manchester Biomedical Research Centre, University of Manchester, Manchester, UK.
  • Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
  • Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK.
  • Centre for Biostatistics, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK.
  • Keele Cardiovascular Research Group, Keele University, Stoke-on-Trent, UK.

ORCID-Profile: Alexander Pate, Matthew Sperrin, Glen P Martin

Förderung

  • Medical Research Council

Zitieren

APA

Pate, A., Sperrin, M., Riley, R. D., Peek, N., Van Staa, T., Sergeant, J. C., Mamas, M. A., Lip, G. Y. H., O'Flaherty, M., Barrowman, M., Buchan, I., & Martin, G. P. (2024). Calibration plots for multistate risk prediction models. Statistics in Medicine, 43(14), 2830-2852. https://doi.org/10.1002/sim.10094

Vancouver

Pate A, Sperrin M, Riley RD, Peek N, Van Staa T, Sergeant JC, et al. Calibration plots for multistate risk prediction models. Statistics in Medicine. 2024;43(14):2830-2852. doi:10.1002/sim.10094

BibTeX
@article{Pate2024,
  author = {Pate, Alexander and Sperrin, Matthew and Riley, Richard D and Peek, Niels and Van Staa, Tjeerd and Sergeant, Jamie C and Mamas, Mamas A and Lip, Gregory Y H and O'Flaherty, Martin and Barrowman, Michael and Buchan, Iain and Martin, Glen P},
  title = {Calibration plots for multistate risk prediction models},
  journal = {Statistics in Medicine},
  year = {2024},
  volume = {43},
  number = {14},
  pages = {2830--2852},
  doi = {10.1002/sim.10094}
}
Kostenlos starten →