Clustered flexible calibration plots for binary outcomes using random effects modeling
Research Synthesis Methods · 2025
DOI: 10.1017/rsm.2025.10046 ↗Kurz gefasst
Fachartikel in der Zeitschrift Research Synthesis Methods, 2025. Verfasst von Lasai Barreñada und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Development and regeneration, https://ror.org/05f950310KU Leuven, Belgium. Hauptthemen laut MEDLINE: Models, Statistical. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: KU Leuven.
Abstract (Originalsprache)
Evaluation of clinical prediction models across multiple clusters, whether centers or datasets, is becoming increasingly common. A comprehensive evaluation includes an assessment of the agreement between the estimated risks and the observed outcomes, also known as calibration. Calibration is of utmost importance for clinical decision making with prediction models, and it often varies between clusters. We present three approaches to take clustering into account when evaluating calibration: (1) clustered group calibration (CG-C), (2) two-stage meta-analysis calibration (2MA-C), and (3) mixed model calibration (MIX-C), which can obtain flexible calibration plots with random effects modeling and provide confidence interval (CI) and prediction interval (PI). As a case example, we externally validate a model to estimate the risk that an ovarian tumor is malignant in multiple centers ( N = 2489). We also conduct a simulation study and a synthetic data study generated from a true clustered dataset to evaluate the methods. In the simulation study, MIX-C and 2MA-C (splines) gave estimated curves closest to the true overall curve. In the synthetic data study, MIX-C produced cluster-specific curves closest to the truth. Coverage of the PI across the plot was best for 2MA-C with splines. We recommend using 2MA-C with splines to estimate the overall curve and 95% PI and MIX-C for cluster-specific curves, especially when the sample size per cluster is limited. We provide ready-to-use code to construct summary flexible calibration curves, with CI and PI to assess heterogeneity in calibration across datasets or centers.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Development and regeneration, https://ror.org/05f950310KU Leuven, Belgium.
- Leuven Unit for Health Technology Assessment Research (LUHTAR), KU Leuven, Belgium.
- Department of Metabolism, Digestion and Reproduction, Imperial College, United Kingdom.
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands.
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands.
- Julius Center, Department of Data Science & Biostatistics, University Medical Center (UMC) Utrecht, Utrecht, the Netherlands.
ORCID-Profile: Lasai Barreñada
Förderung
- KU Leuven
Zitieren
APA
Barreñada, L., De Cock Campo, B., Wynants, L., & Van Calster, B. (2025). Clustered flexible calibration plots for binary outcomes using random effects modeling. Research Synthesis Methods. https://doi.org/10.1017/rsm.2025.10046
Vancouver
Barreñada L, De Cock Campo B, Wynants L, Van Calster B. Clustered flexible calibration plots for binary outcomes using random effects modeling. Research Synthesis Methods. 2025. doi:10.1017/rsm.2025.10046
BibTeX
@article{Barreada2025,
author = {Barreñada, Lasai and De Cock Campo, Bavo and Wynants, Laure and Van Calster, Ben},
title = {Clustered flexible calibration plots for binary outcomes using random effects modeling},
journal = {Research Synthesis Methods},
year = {2025},
doi = {10.1017/rsm.2025.10046}
}
EUVIMED