Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees
Frontiers in Digital Health · 24.04.2025 · 7
DOI: 10.3389/fdgth.2025.1576290 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Digital Health, online erschienen am 24.04.2025. Verfasst von Mikel Hernandez und 5 weiteren Autor:innen. Beteiligte Einrichtung: Digital Health and Biomedical Technologies, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain. Die Arbeit wurde bisher 47-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The generation of synthetic tabular data has emerged as a key privacy-enhancing technology to address challenges in data sharing, particularly in healthcare, where sensitive attributes can compromise patient privacy. Despite significant progress, balancing fidelity, utility, and privacy in complex medical datasets remains a substantial challenge. This paper introduces a comprehensive and holistic evaluation framework for synthetic tabular data, consolidating metrics and privacy risk measures across three key categories (fidelity, utility and privacy) and incorporating a fidelity-utility tradeoff metric. The framework was applied to three open-source medical datasets to evaluate synthetic tabular data generated by five generative models, both with and without differential privacy. Results showed that simpler models generally achieved better fidelity and utility, while more complex models provided lower privacy risks. The addition of differential privacy enhanced privacy preservation but often reduced fidelity and utility, highlighting the complexity of balancing fidelity, utility and privacy in synthetic data generation for medical datasets. Despite its contributions, this study acknowledges limitations, such as the lack of evaluation metrics neither privacy risk measures for required model training time and resource usage, reliance on default model parameters, and the assessment of models that incorporates differential privacy with only a single privacy budget. Future work should explore parameter optimization, alternative privacy mechanisms, broader applications of the framework to diverse datasets and domains, and collaborations with clinicians for clinical utility evaluation. This study provides a foundation for improving synthetic tabular data evaluation and advancing privacy-preserving data sharing in healthcare.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Digital Health and Biomedical Technologies, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastián, Spain.
- Computer Science and Artificial Intelligence Department, Computer Science Faculty, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain.
- eHealth Group, Biogipuzkoa Health Research Institute, Donostia-San Sebastián, Spain.
- School of Applied Sciences and Engineering, Universidad EAFIT, Medellín, Colombia.
Zitieren
APA
Hernandez, M., Osorio-Marulanda, P. A., Catalina, M., Loinaz, L., Epelde, G., & Aginako, N. (2025). Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees. Frontiers in Digital Health, 7. https://doi.org/10.3389/fdgth.2025.1576290
Vancouver
Hernandez M, Osorio-Marulanda PA, Catalina M, Loinaz L, Epelde G, Aginako N. Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees. Frontiers in Digital Health. 2025;7. doi:10.3389/fdgth.2025.1576290
BibTeX
@article{Hernandez2025,
author = {Hernandez, Mikel and Osorio-Marulanda, Pablo A and Catalina, Mikel and Loinaz, Lorea and Epelde, Gorka and Aginako, Naiara},
title = {Comprehensive evaluation framework for synthetic tabular data in health: fidelity, utility and privacy analysis of generative models with and without privacy guarantees},
journal = {Frontiers in Digital Health},
year = {2025},
volume = {7},
doi = {10.3389/fdgth.2025.1576290}
}
EUVIMED