EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Estimating and Evaluating Counterfactual Prediction Models

Christopher B Boyer iD, Issa J Dahabreh, Jon A Steingrimsson

Statistics in Medicine · 2025-10 · 44 (23-24)

DOI: 10.1002/sim.70287 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Statistics in Medicine, online erschienen am 07.10.2025. Verfasst von Christopher B Boyer und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Quantitative Health Sciences, Cleveland Clinic Research, Cleveland, Ohio, USA. Hauptthemen laut MEDLINE: Models, Statistical. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: NIH HHS, U.S. National Library of Medicine, NLM NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Counterfactual prediction methods are required when a model will be deployed in a setting where treatment policies differ from the setting where the model was developed, or when a model provides predictions under hypothetical interventions to support decision‐making. However, estimating and evaluating counterfactual prediction models is challenging because, unlike traditional (factual) prediction, one does not observe the potential outcomes for all individuals under all treatment strategies of interest. Here, we discuss how to estimate a counterfactual prediction model, how to assess the model's performance, and how to perform model and tuning parameter selection. We provide identification and estimation results for counterfactual prediction models and for multiple measures of counterfactual model performance, including loss‐based measures, the area under the receiver operating characteristic curve, and the calibration curve. Importantly, our results allow valid estimates of model performance under counterfactual intervention even if the candidate prediction model is misspecified, permitting a wider array of use cases. We illustrate these methods using simulation and apply them to the task of developing a statin‐naïve risk prediction model for cardiovascular disease.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Models, StatisticalHumansComputer SimulationCardiovascular DiseasesROC CurveRisk AssessmentHydroxymethylglutaryl-CoA Reductase InhibitorsArea Under Curve

Stichwörter der Autor:innen

causal inferencemachine learningmodel performanceprediction modeltransportabilitytreatment drop‐in

Substanzen

Hydroxymethylglutaryl-CoA Reductase Inhibitors

Autor:innen und Institutionen

  • Department of Quantitative Health Sciences, Cleveland Clinic Research, Cleveland, Ohio, USA.
  • Department of Medicine, Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, Ohio, USA.
  • Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA.
  • CAUSALab, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA.
  • Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA.
  • Richard A. and Susan F. Smith Center for Outcomes Research, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA.

ORCID-Profile: Christopher B Boyer

Förderung

  • NIH HHS
  • U.S. National Library of Medicine
  • NLM NIH HHS
  • NHLBI NIH HHS
  • Patient-Centered Outcomes Research Institute
  • National Institutes of Health
  • National Heart, Lung, and Blood Institute

Zitieren

APA

Boyer, C. B., Dahabreh, I. J., & Steingrimsson, J. A. (2025). Estimating and Evaluating Counterfactual Prediction Models. Statistics in Medicine, 44(23-24). https://doi.org/10.1002/sim.70287

Vancouver

Boyer CB, Dahabreh IJ, Steingrimsson JA. Estimating and Evaluating Counterfactual Prediction Models. Statistics in Medicine. 2025;44(23-24). doi:10.1002/sim.70287

BibTeX
@article{Boyer2025,
  author = {Boyer, Christopher B and Dahabreh, Issa J and Steingrimsson, Jon A},
  title = {Estimating and Evaluating Counterfactual Prediction Models},
  journal = {Statistics in Medicine},
  year = {2025},
  volume = {44},
  number = {23-24},
  doi = {10.1002/sim.70287}
}
Kostenlos starten →