Bayesian Sample Size Calculations for External Validation Studies of Risk Prediction Models
Statistics in Medicine · 2026-02 · 45 (3-5)
DOI: 10.1002/sim.70389 ↗Kurz gefasst
Fachartikel in der Zeitschrift Statistics in Medicine, online erschienen am 11.02.2026. Verfasst von Mohsen Sadatsafavi und 4 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Pharmaceutical Sciences and Faculty of Medicine, The University of British Columbia, Vancouver, British Columbia, Canada. Hauptthemen laut MEDLINE: Models, Statistical. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: CIHR, Natural Sciences and Engineering Research Council of Canada, Talent Programme financed by the Dutch Research.
Abstract (Originalsprache)
Contemporary sample size calculations for external validation of risk prediction models require users to specify fixed values of assumed model performance metrics alongside target precision levels (e.g., 95% CI widths). However, due to the finite samples of previous studies, our knowledge of true model performance in the target population is uncertain, and so choosing fixed values represents an incomplete picture. As well, for net benefit (NB) as a measure of clinical utility, the relevance of conventional precision‐based inference is doubtful. In this work, we propose a general Bayesian framework for multi‐criteria sample size considerations for prediction models for binary outcomes. For statistical metrics of performance (e.g., discrimination and calibration), we propose sample size rules that target desired expected precision or desired assurance probability that the precision criteria will be satisfied. For NB, we propose rules based on Optimality Assurance (the probability that the planned study correctly identifies the optimal strategy) and Value of Information (VoI) analysis, which quantifies the expected gain in NB by learning about model performance from a validation study of a given size. We showcase these developments in a case study on the validation of a risk prediction model for deterioration among hospitalized COVID‐19 patients. Compared to conventional sample size calculation methods, a Bayesian approach requires explicit quantification of uncertainty around model performance, and thereby enables flexible sample size rules based on expected precision, assurance probabilities, and VoI. In our case study, calculations based on VoI for NB suggest considerably lower sample sizes are required than when focusing on the precision of calibration metrics. This approach is implemented in the accompanying software.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Pharmaceutical Sciences and Faculty of Medicine, The University of British Columbia, Vancouver, British Columbia, Canada.
- Department of Statistics, The University of British Columbia, Vancouver, British Columbia, Canada.
- British Columbia Centre For Disease Control, Vancouver, British Columbia, Canada.
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, the Netherlands.
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium.
- Department of Applied Health Sciences, School of Health Sciences, College of Medicine and Health, University of Birmingham, Birmingham, UK.
ORCID-Profile: Mohsen Sadatsafavi, Paul Gustafson, Laure Wynants, Richard D Riley
Förderung
- CIHR
- Natural Sciences and Engineering Research Council of Canada
- Talent Programme financed by the Dutch Research
- UK Engineering and Physical Sciences Research Council
- UK Medical Research Council - National Institute for Health and Care Research Better Methods Better Research
- Canadian Institutes of Health Research
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Birmingham Biomedical Research Centre
Zitieren
APA
Sadatsafavi, M., Gustafson, P., Setayeshgar, S., Wynants, L., & D Riley, R. (2026). Bayesian Sample Size Calculations for External Validation Studies of Risk Prediction Models. Statistics in Medicine, 45(3-5). https://doi.org/10.1002/sim.70389
Vancouver
Sadatsafavi M, Gustafson P, Setayeshgar S, Wynants L, D Riley R. Bayesian Sample Size Calculations for External Validation Studies of Risk Prediction Models. Statistics in Medicine. 2026;45(3-5). doi:10.1002/sim.70389
BibTeX
@article{Sadatsafavi2026,
author = {Sadatsafavi, Mohsen and Gustafson, Paul and Setayeshgar, Solmaz and Wynants, Laure and D Riley, Richard},
title = {Bayesian Sample Size Calculations for External Validation Studies of Risk Prediction Models},
journal = {Statistics in Medicine},
year = {2026},
volume = {45},
number = {3-5},
doi = {10.1002/sim.70389}
}
EUVIMED