Robust privacy amidst innovation with large language models through a critical assessment of the risks
Journal of the American Medical Informatics Association · 2025
DOI: 10.1093/jamia/ocaf037 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2025. Verfasst von Yao-Shun Chuang und 4 weiteren Autor:innen. Beteiligte Einrichtung: McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States. Hauptthemen laut MEDLINE: Natural Language Processing, Electronic Health Records, Confidentiality, Computer Security. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCI NIH HHS, NSERC Discovery Grants, National Science Foundation.
Abstract (Originalsprache)
Objective
This study evaluates the integration of electronic health records (EHRs) and natural language processing (NLP) with large language models (LLMs) to enhance healthcare data management and patient care, focusing on using advanced language models to create secure, Health Insurance Portability and Accountability Act-compliant synthetic patient notes for global biomedical research. Materials and Methods The study used de-identified and re-identified versions of the MIMIC III dataset with GPT-3.5, GPT-4, and Mistral 7B to generate synthetic clinical notes. Text generation employed templates and keyword extraction for contextually relevant notes, with One-shot generation for comparison. Privacy was assessed by analyzing protected health information (PHI) occurrence and co-occurrence, while utility was evaluated by training an ICD-9 coder using synthetic notes. Text quality was measured using ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and cosine similarity metrics to compare synthetic notes with source notes for semantic similarity.
Results
The analysis of PHI occurrence and text utility via the ICD-9 coding task showed that the keyword-based method had low risk and good performance. One-shot generation exhibited the highest PHI exposure and PHI co-occurrence, particularly in geographic location and date categories. The Normalized One-shot method achieved the highest classification accuracy. Re-identified data consistently outperformed de-identified data.
Discussion
Privacy analysis revealed a critical balance between data utility and privacy protection, influencing future data use and sharing.
Conclusion
This study shows that keyword-based methods can create synthetic clinical notes that protect privacy while retaining data usability, potentially improving clinical data sharing. The use of dummy PHIs to counter privacy attacks may offer better utility and privacy than traditional de-identification.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States.
- Department of Computer Science, University of Manitoba, Winnipeg, Manitoba R3T 5V6, Canada.
ORCID-Profile: Yu-Chun Hsu, Xiaoqian Jiang
Förderung
- NCI NIH HHS
- NSERC Discovery Grants
- National Science Foundation
- NIA NIH HHS
- Cancer Research
- NCATS NIH HHS
- NSERC
- NLM NIH HHS
Zitieren
APA
Chuang, Y. S., Sarkar, A. R., Hsu, Y. C., Mohammed, N., & Jiang, X. (2025). Robust privacy amidst innovation with large language models through a critical assessment of the risks. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaf037
Vancouver
Chuang YS, Sarkar AR, Hsu YC, Mohammed N, Jiang X. Robust privacy amidst innovation with large language models through a critical assessment of the risks. Journal of the American Medical Informatics Association. 2025. doi:10.1093/jamia/ocaf037
BibTeX
@article{Chuang2025,
author = {Chuang, Yao-Shun and Sarkar, Atiquer Rahman and Hsu, Yu-Chun and Mohammed, Noman and Jiang, Xiaoqian},
title = {Robust privacy amidst innovation with large language models through a critical assessment of the risks},
journal = {Journal of the American Medical Informatics Association},
year = {2025},
doi = {10.1093/jamia/ocaf037}
}
EUVIMED