Does Differentially Private Synthetic Data Lead to Synthetic Discoveries?
Methods of Information in Medicine · 01.05.2024 · 63 (01/02) · 035-051
DOI: 10.1055/a-2385-1355 ↗Kurz gefasst
Fachartikel in der Zeitschrift Methods of Information in Medicine, erschienen am 01.05.2024. Verfasst von Ileana Montoya Perez und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computing, University of Turku, Turku, Finland. Hauptthemen laut MEDLINE: Algorithms. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Business Finland, European Union's Horizon Europe research and innovation.
Abstract (Originalsprache)
Background
Synthetic data have been proposed as a solution for sharing anonymized versions of sensitive biomedical datasets. Ideally, synthetic data should preserve the structure and statistical properties of the original data, while protecting the privacy of the individual subjects. Differential Privacy (DP) is currently considered the gold standard approach for balancing this trade-off.
Objectives
The aim of this study is to investigate how trustworthy are group differences discovered by independent sample tests from DP-synthetic data. The evaluation is carried out in terms of the tests' Type I and Type II errors. With the former, we can quantify the tests' validity, i.e., whether the probability of false discoveries is indeed below the significance level, and the latter indicates the tests' power in making real discoveries.
Methods
We evaluate the Mann–Whitney U test, Student's t-test, chi-squared test, and median test on DP-synthetic data. The private synthetic datasets are generated from real-world data, including a prostate cancer dataset (n = 500) and a cardiovascular dataset (n = 70,000), as well as on bivariate and multivariate simulated data. Five different DP-synthetic data generation methods are evaluated, including two basic DP histogram release methods and MWEM, Private-PGM, and DP GAN algorithms.
Conclusion
A large portion of the evaluation results expressed dramatically inflated Type I errors, especially at levels of ϵ ≤ 1. This result calls for caution when releasing and analyzing DP-synthetic data: low p-values may be obtained in statistical tests simply as a byproduct of the noise added to protect privacy. A DP Smoothed Histogram-based synthetic data generation method was shown to produce valid Type I error for all privacy levels tested but required a large original dataset size and a modest privacy budget (ϵ ≥ 5) in order to have reasonable Type II error levels.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Computing, University of Turku, Turku, Finland.
Förderung
- Business Finland
- European Union's Horizon Europe research and innovation
Zitieren
APA
Montoya Perez, I., Movahedi, P., Nieminen, V., Airola, A., & Pahikkala, T. (2024). Does Differentially Private Synthetic Data Lead to Synthetic Discoveries?. Methods of Information in Medicine, 63(01/02), 035-051. https://doi.org/10.1055/a-2385-1355
Vancouver
Montoya Perez I, Movahedi P, Nieminen V, Airola A, Pahikkala T. Does Differentially Private Synthetic Data Lead to Synthetic Discoveries?. Methods of Information in Medicine. 2024;63(01/02):035-051. doi:10.1055/a-2385-1355
BibTeX
@article{MontoyaPerez2024,
author = {Montoya Perez, Ileana and Movahedi, Parisa and Nieminen, Valtteri and Airola, Antti and Pahikkala, Tapio},
title = {Does Differentially Private Synthetic Data Lead to Synthetic Discoveries?},
journal = {Methods of Information in Medicine},
year = {2024},
volume = {63},
number = {01/02},
pages = {035--051},
doi = {10.1055/a-2385-1355}
}
EUVIMED