Non‐parametric inference on calibration of predicted risks
Statistics in Medicine · 2024
DOI: 10.1002/sim.10138 ↗At a glance
Journal article in Statistics in Medicine, 2024. Written by Mohsen Sadatsafavi and 1 co-authors. Institution: Faculty of Pharmaceutical Sciences and Faculty of Medicine, The University of British Columbia, Vancouver, British Columbia, Canada. Major MEDLINE topics: Models, Statistical, Computer Simulation. Cited 7 times so far (Crossref); 34 references. The full text is freely available (open access, CC BY 4.0). Funded by: CIHR, Canadian Institutes of Health Research.
Abstract (original language)
Moderate calibration, the expected event probability among observations with predicted probability z being equal to z, is a desired property of risk prediction models. Current graphical and numerical techniques for evaluating moderate calibration of risk prediction models are mostly based on smoothing or grouping the data. As well, there is no widely accepted inferential method for the null hypothesis that a model is moderately calibrated. In this work, we discuss recently‐developed, and propose novel, methods for the assessment of moderate calibration for binary responses. The methods are based on the limiting distributions of functions of standardized partial sums of prediction errors converging to the corresponding laws of Brownian motion. The novel method relies on well‐known properties of the Brownian bridge which enables joint inference on mean and moderate calibration, leading to a unified “bridge” test for detecting miscalibration. Simulation studies indicate that the bridge test is more powerful, often substantially, than the alternative test. As a case study we consider a prediction model for short‐term mortality after a heart attack, where we provide suggestions on graphical presentation and the interpretation of results. Moderate calibration can be assessed without requiring arbitrary grouping of data or using methods that require tuning of parameters.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Faculty of Pharmaceutical Sciences and Faculty of Medicine, The University of British Columbia, Vancouver, British Columbia, Canada.
- Department of Statistics, The University of British Columbia, Vancouver, British Columbia, Canada.
ORCID profiles: Mohsen Sadatsafavi
Funding
- CIHR
- Canadian Institutes of Health Research
Cite
APA
Sadatsafavi, M., & Petkau, J. (2024). Non‐parametric inference on calibration of predicted risks. Statistics in Medicine. https://doi.org/10.1002/sim.10138
Vancouver
Sadatsafavi M, Petkau J. Non‐parametric inference on calibration of predicted risks. Statistics in Medicine. 2024. doi:10.1002/sim.10138
BibTeX
@article{Sadatsafavi2024,
author = {Sadatsafavi, Mohsen and Petkau, John},
title = {Non‐parametric inference on calibration of predicted risks},
journal = {Statistics in Medicine},
year = {2024},
doi = {10.1002/sim.10138}
}
EUVIMED