EUVIMEDEUVIMED entdecken
Crossref

Quality metrics of synthetic radiomics data do not predict improvement under simulated external validation: an ecological fallacy across 50 public datasets

Clemente García-Hidalgo, José Antonio Consentino Hernández, José Vicente Cayuela Espí, Gonzalo Pagán Vicente, Juan Francisco Mula Ponce, Ángel Vázquez Olmos

European Radiology · 2026

DOI: 10.1007/s00330-026-12703-4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift European Radiology, 2026. Verfasst von Clemente García-Hidalgo und 5 weiteren Autor:innen. Beteiligte Einrichtung: Radiology Department, Hospital Universitario Morales Meseguer, Murcia, Spain. clemente292@gmail.com. Hauptthemen laut MEDLINE: Radiomics. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 29 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To investigate whether established synthetic data quality metrics predict when deep generative augmentation improves performance under simulated external validation in radiomics. Materials and methods Three conditional generators (WGAN-GP, CVAE, TabDDPM) were trained on 50 public binary-classification radiomic datasets from radMLBench. For each dataset–generator pair ( n = 150), five quality metrics and ΔAUC (change in simulated external AUC under domain shift) were recorded across 10 repetitions; undefined AUCs (single-class test folds) were excluded a priori (30/2500 external, 1.2%). Six analyses tested quality–ΔAUC associations: rank correlations with FDR correction, ROC discrimination, subgroup random-effects meta-analysis, composite scoring, quality-guided generator selection, and SMOTE comparison. All experiments were replicated with a random forest classifier.

Results

Pooled quality metrics correlated significantly with ΔAUC (|ρ| = 0.30–0.36, all adjusted p 0.05), revealing an ecological fallacy. After Benjamini–Hochberg correction, 5 of 20 correlations survived: the 4 pooled associations and DDPM MMD (unexpected direction). ROC-AUC ranged from 0.42 to 0.61 (near-chance). Quality-guided selection yielded a significantly negative pooled Δ (−0.0051; 95% CI: −0.0083 to −0.0019) and improved external AUC in only 17/50 (34%) datasets versus 34/50 (68%) for an oracle. Random forest replication rendered pooled correlations non-significant (all p > 0.09). A pre-specified sensitivity analysis on the 29/50 datasets statistically distinguishable from chance produced identical conclusions.

Conclusion

Aggregate quality–performance correlations are driven by between-generator differences, not within-generator variation that could guide practical decisions. Quality metrics are insufficient proxies for clinical utility; task-specific external validation remains indispensable. Key Points Question Can established quality metrics of synthetic radiomics data identify when generative augmentation actually improves the performance of prediction models under simulated external validation?

Findings

Across 50 public datasets, aggregate quality–performance correlations vanished within individual generators (ecological fallacy), and quality-guided generator selection underperformed real-only training. Clinical relevance Quality metrics commonly used to validate synthetic radiomics data cannot identify when augmentation improves clinical model performance under simulated domain shift, underscoring the irreplaceable role of task-specific external validation before clinical deployment. Graphical Abstract

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

RadiomicsHumansReproducibility of ResultsComputer SimulationROC CurveArea Under CurveGenerative Artificial IntelligenceData Accuracy

Stichwörter der Autor:innen

Computer-assistedDeep learningModels, statisticalRadiomicsReproducibility of results

Autor:innen und Institutionen

  • Radiology Department, Hospital Universitario Morales Meseguer, Murcia, Spain. clemente292@gmail.com.
  • Radiology Department, Hospital Universitario Morales Meseguer, Murcia, Spain.

Zitieren

APA

García-Hidalgo, C., Consentino Hernández, J. A., Cayuela Espí, J. V., Pagán Vicente, G., Mula Ponce, J. F., & Vázquez Olmos, Á. (2026). Quality metrics of synthetic radiomics data do not predict improvement under simulated external validation: an ecological fallacy across 50 public datasets. European Radiology. https://doi.org/10.1007/s00330-026-12703-4

Vancouver

García-Hidalgo C, Consentino Hernández JA, Cayuela Espí JV, Pagán Vicente G, Mula Ponce JF, Vázquez Olmos Á. Quality metrics of synthetic radiomics data do not predict improvement under simulated external validation: an ecological fallacy across 50 public datasets. European Radiology. 2026. doi:10.1007/s00330-026-12703-4

BibTeX
@article{GarcaHidalgo2026,
  author = {García-Hidalgo, Clemente and Consentino Hernández, José Antonio and Cayuela Espí, José Vicente and Pagán Vicente, Gonzalo and Mula Ponce, Juan Francisco and Vázquez Olmos, Ángel},
  title = {Quality metrics of synthetic radiomics data do not predict improvement under simulated external validation: an ecological fallacy across 50 public datasets},
  journal = {European Radiology},
  year = {2026},
  doi = {10.1007/s00330-026-12703-4}
}
Kostenlos starten →