EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Clustered flexible calibration plots for binary outcomes using random effects modeling

Lasai Barreñada iD, Bavo De Cock Campo, Laure Wynants, Ben Van Calster

Research Synthesis Methods · 2025

DOI: 10.1017/rsm.2025.10046 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Research Synthesis Methods, 2025. Verfasst von Lasai Barreñada und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Development and regeneration, https://ror.org/05f950310KU Leuven, Belgium. Hauptthemen laut MEDLINE: Models, Statistical. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: KU Leuven.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Evaluation of clinical prediction models across multiple clusters, whether centers or datasets, is becoming increasingly common. A comprehensive evaluation includes an assessment of the agreement between the estimated risks and the observed outcomes, also known as calibration. Calibration is of utmost importance for clinical decision making with prediction models, and it often varies between clusters. We present three approaches to take clustering into account when evaluating calibration: (1) clustered group calibration (CG-C), (2) two-stage meta-analysis calibration (2MA-C), and (3) mixed model calibration (MIX-C), which can obtain flexible calibration plots with random effects modeling and provide confidence interval (CI) and prediction interval (PI). As a case example, we externally validate a model to estimate the risk that an ovarian tumor is malignant in multiple centers ( N = 2489). We also conduct a simulation study and a synthetic data study generated from a true clustered dataset to evaluate the methods. In the simulation study, MIX-C and 2MA-C (splines) gave estimated curves closest to the true overall curve. In the synthetic data study, MIX-C produced cluster-specific curves closest to the truth. Coverage of the PI across the plot was best for 2MA-C with splines. We recommend using 2MA-C with splines to estimate the overall curve and 95% PI and MIX-C for cluster-specific curves, especially when the sample size per cluster is limited. We provide ready-to-use code to construct summary flexible calibration curves, with CI and PI to assess heterogeneity in calibration across datasets or centers.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Models, StatisticalCalibrationHumansCluster AnalysisComputer SimulationFemaleAlgorithmsOvarian NeoplasmsReproducibility of ResultsPrediction AlgorithmsMeta-Analysis as TopicClustering AlgorithmsConfidence Intervals

Stichwörter der Autor:innen

Binary OutcomesCalibrationClustered DataMeta-analysisModel validationprediction models

Autor:innen und Institutionen

  • Department of Development and regeneration, https://ror.org/05f950310KU Leuven, Belgium.
  • Leuven Unit for Health Technology Assessment Research (LUHTAR), KU Leuven, Belgium.
  • Department of Metabolism, Digestion and Reproduction, Imperial College, United Kingdom.
  • Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands.
  • Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands.
  • Julius Center, Department of Data Science & Biostatistics, University Medical Center (UMC) Utrecht, Utrecht, the Netherlands.

ORCID-Profile: Lasai Barreñada

Förderung

  • KU Leuven

Zitieren

APA

Barreñada, L., De Cock Campo, B., Wynants, L., & Van Calster, B. (2025). Clustered flexible calibration plots for binary outcomes using random effects modeling. Research Synthesis Methods. https://doi.org/10.1017/rsm.2025.10046

Vancouver

Barreñada L, De Cock Campo B, Wynants L, Van Calster B. Clustered flexible calibration plots for binary outcomes using random effects modeling. Research Synthesis Methods. 2025. doi:10.1017/rsm.2025.10046

BibTeX
@article{Barreada2025,
  author = {Barreñada, Lasai and De Cock Campo, Bavo and Wynants, Laure and Van Calster, Ben},
  title = {Clustered flexible calibration plots for binary outcomes using random effects modeling},
  journal = {Research Synthesis Methods},
  year = {2025},
  doi = {10.1017/rsm.2025.10046}
}
Kostenlos starten →