EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees

Mikel Hernandez, Pablo A Osorio-Marulanda, Mikel Catalina, Lorea Loinaz, Gorka Epelde, Naiara Aginako

Frontiers in Digital Health · 24.04.2025 · 7

DOI: 10.3389/fdgth.2025.1576290 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Digital Health, online erschienen am 24.04.2025. Verfasst von Mikel Hernandez und 5 weiteren Autor:innen. Beteiligte Einrichtung: Digital Health and Biomedical Technologies, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain. Die Arbeit wurde bisher 47-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The generation of synthetic tabular data has emerged as a key privacy-enhancing technology to address challenges in data sharing, particularly in healthcare, where sensitive attributes can compromise patient privacy. Despite significant progress, balancing fidelity, utility, and privacy in complex medical datasets remains a substantial challenge. This paper introduces a comprehensive and holistic evaluation framework for synthetic tabular data, consolidating metrics and privacy risk measures across three key categories (fidelity, utility and privacy) and incorporating a fidelity-utility tradeoff metric. The framework was applied to three open-source medical datasets to evaluate synthetic tabular data generated by five generative models, both with and without differential privacy. Results showed that simpler models generally achieved better fidelity and utility, while more complex models provided lower privacy risks. The addition of differential privacy enhanced privacy preservation but often reduced fidelity and utility, highlighting the complexity of balancing fidelity, utility and privacy in synthetic data generation for medical datasets. Despite its contributions, this study acknowledges limitations, such as the lack of evaluation metrics neither privacy risk measures for required model training time and resource usage, reliance on default model parameters, and the assessment of models that incorporates differential privacy with only a single privacy budget. Future work should explore parameter optimization, alternative privacy mechanisms, broader applications of the framework to diverse datasets and domains, and collaborations with clinicians for clinical utility evaluation. This study provides a foundation for improving synthetic tabular data evaluation and advancing privacy-preserving data sharing in healthcare.

Schlagwörter

Stichwörter der Autor:innen

differential privacygenerative modelsprivacy risk measureattackssynthetic data evaluationsynthetic data fidelitysynthetic data generationsynthetic data privacysynthetic data utility

Autor:innen und Institutionen

  • Digital Health and Biomedical Technologies, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain.
  • Computer Science and Artificial Intelligence Department, Computer Science Faculty, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain.
  • eHealth Group, Biogipuzkoa Health Research Institute, Donostia-San Sebastián, Spain.
  • School of Applied Sciences and Engineering, Universidad EAFIT, Medellín, Colombia.

Zitieren

APA

Hernandez, M., Osorio-Marulanda, P. A., Catalina, M., Loinaz, L., Epelde, G., & Aginako, N. (2025). Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees. Frontiers in Digital Health, 7. https://doi.org/10.3389/fdgth.2025.1576290

Vancouver

Hernandez M, Osorio-Marulanda PA, Catalina M, Loinaz L, Epelde G, Aginako N. Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees. Frontiers in Digital Health. 2025;7. doi:10.3389/fdgth.2025.1576290

BibTeX
@article{Hernandez2025,
  author = {Hernandez, Mikel and Osorio-Marulanda, Pablo A and Catalina, Mikel and Loinaz, Lorea and Epelde, Gorka and Aginako, Naiara},
  title = {Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees},
  journal = {Frontiers in Digital Health},
  year = {2025},
  volume = {7},
  doi = {10.3389/fdgth.2025.1576290}
}
Kostenlos starten →