EUVIMEDSkopri EUVIMED
Open access · CC BY 4.0

Non‐parametric inference on calibration of predicted risks

Mohsen Sadatsafavi iD, John Petkau

Statistics in Medicine · 2024

DOI: 10.1002/sim.10138 ↗

At a glance

Journal article in Statistics in Medicine, 2024. Written by Mohsen Sadatsafavi and 1 co-authors. Institution: Faculty of Pharmaceutical Sciences and Faculty of Medicine, The University of British Columbia, Vancouver, British Columbia, Canada. Major MEDLINE topics: Models, Statistical, Computer Simulation. Cited 7 times so far (Crossref); 34 references. The full text is freely available (open access, CC BY 4.0). Funded by: CIHR, Canadian Institutes of Health Research.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Moderate calibration, the expected event probability among observations with predicted probability z being equal to z, is a desired property of risk prediction models. Current graphical and numerical techniques for evaluating moderate calibration of risk prediction models are mostly based on smoothing or grouping the data. As well, there is no widely accepted inferential method for the null hypothesis that a model is moderately calibrated. In this work, we discuss recently‐developed, and propose novel, methods for the assessment of moderate calibration for binary responses. The methods are based on the limiting distributions of functions of standardized partial sums of prediction errors converging to the corresponding laws of Brownian motion. The novel method relies on well‐known properties of the Brownian bridge which enables joint inference on mean and moderate calibration, leading to a unified “bridge” test for detecting miscalibration. Simulation studies indicate that the bridge test is more powerful, often substantially, than the alternative test. As a case study we consider a prediction model for short‐term mortality after a heart attack, where we provide suggestions on graphical presentation and the interpretation of results. Moderate calibration can be assessed without requiring arbitrary grouping of data or using methods that require tuning of parameters.

Subjects

MeSH terms (MEDLINE)

Models, StatisticalComputer SimulationHumansRisk AssessmentMyocardial InfarctionStatistics, NonparametricCalibrationProbability

Author keywords

calibrationhypothesis testinginferencerisk prediction

Authors and institutions

  • Faculty of Pharmaceutical Sciences and Faculty of Medicine, The University of British Columbia, Vancouver, British Columbia, Canada.
  • Department of Statistics, The University of British Columbia, Vancouver, British Columbia, Canada.

ORCID profiles: Mohsen Sadatsafavi

Funding

  • CIHR
  • Canadian Institutes of Health Research

Cite

APA

Sadatsafavi, M., & Petkau, J. (2024). Non‐parametric inference on calibration of predicted risks. Statistics in Medicine. https://doi.org/10.1002/sim.10138

Vancouver

Sadatsafavi M, Petkau J. Non‐parametric inference on calibration of predicted risks. Statistics in Medicine. 2024. doi:10.1002/sim.10138

BibTeX
@article{Sadatsafavi2024,
  author = {Sadatsafavi, Mohsen and Petkau, John},
  title = {Non‐parametric inference on calibration of predicted risks},
  journal = {Statistics in Medicine},
  year = {2024},
  doi = {10.1002/sim.10138}
}
Start for free →