EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Evaluating individualized treatment effect predictions: A model‐based perspective on discrimination and calibration assessment

J Hoogland iD, O Efthimiou iD, T L Nguyen iD, T P A Debray iD

Statistics in Medicine · 2024

DOI: 10.1002/sim.10186 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Statistics in Medicine, 2024. Verfasst von J Hoogland und 3 weiteren Autor:innen. Beteiligte Einrichtung: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands. Hauptthemen laut MEDLINE: Models, Statistical, Randomized Controlled Trials as Topic, Stroke, Computer Simulation. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung, European Union's Horizon 2020 research and innovation program, ZonMw.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, there has been a growing interest in the prediction of individualized treatment effects. While there is a rapidly growing literature on the development of such models, there is little literature on the evaluation of their performance. In this paper, we aim to facilitate the validation of prediction models for individualized treatment effects. The estimands of interest are defined based on the potential outcomes framework, which facilitates a comparison of existing and novel measures. In particular, we examine existing measures of discrimination for benefit (variations of the c‐for‐benefit), and propose model‐based extensions to the treatment effect setting for discrimination and calibration metrics that have a strong basis in outcome risk prediction. The main focus is on randomized trial data with binary endpoints and on models that provide individualized treatment effect predictions and potential outcome predictions. We use simulated data to provide insight into the characteristics of the examined discrimination and calibration statistics under consideration, and further illustrate all methods in a trial of acute ischemic stroke treatment. The results show that the proposed model‐based statistics had the best characteristics in terms of bias and accuracy. While resampling methods adjusted for the optimism of performance estimates in the development data, they had a high variance across replications that limited their accuracy. Therefore, individualized treatment effect models are best validated in independent data. To aid implementation, a software implementation of the proposed methods was made available in R .

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Models, StatisticalRandomized Controlled Trials as TopicStrokeComputer SimulationHumansCalibrationTreatment OutcomePrecision Medicine

Stichwörter der Autor:innen

calibrationdiscriminationindividualizedpredictiontreatment effect

Autor:innen und Institutionen

  • Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
  • Epidemiology and Data Science, Amsterdam University Medical Center, Amsterdam, The Netherlands.
  • Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland.
  • Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland.
  • Section of Epidemiology, Department of Public Health, University of Copenhagen, Copenhagen, Denmark.
  • Smart Data Analysis and Statistics B.V., Utrecht, The Netherlands.

ORCID-Profile: J Hoogland, O Efthimiou, T L Nguyen, T P A Debray

Förderung

  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
  • European Union's Horizon 2020 research and innovation program
  • ZonMw

Zitieren

APA

Hoogland, J., Efthimiou, O., Nguyen, T. L., & Debray, T. P. A. (2024). Evaluating individualized treatment effect predictions: A model‐based perspective on discrimination and calibration assessment. Statistics in Medicine. https://doi.org/10.1002/sim.10186

Vancouver

Hoogland J, Efthimiou O, Nguyen TL, Debray TPA. Evaluating individualized treatment effect predictions: A model‐based perspective on discrimination and calibration assessment. Statistics in Medicine. 2024. doi:10.1002/sim.10186

BibTeX
@article{Hoogland2024,
  author = {Hoogland, J and Efthimiou, O and Nguyen, T L and Debray, T P A},
  title = {Evaluating individualized treatment effect predictions: A model‐based perspective on discrimination and calibration assessment},
  journal = {Statistics in Medicine},
  year = {2024},
  doi = {10.1002/sim.10186}
}
Kostenlos starten →