EUVIMEDEUVIMED entdecken
Open Access

Robust privacy amidst innovation with large language models through a critical assessment of the risks

Yao-Shun Chuang, Atiquer Rahman Sarkar, Yu-Chun Hsu iD, Noman Mohammed, Xiaoqian Jiang iD

Journal of the American Medical Informatics Association · 2025

DOI: 10.1093/jamia/ocaf037 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2025. Verfasst von Yao-Shun Chuang und 4 weiteren Autor:innen. Beteiligte Einrichtung: McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States. Hauptthemen laut MEDLINE: Natural Language Processing, Electronic Health Records, Confidentiality, Computer Security. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCI NIH HHS, NSERC Discovery Grants, National Science Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

This study evaluates the integration of electronic health records (EHRs) and natural language processing (NLP) with large language models (LLMs) to enhance healthcare data management and patient care, focusing on using advanced language models to create secure, Health Insurance Portability and Accountability Act-compliant synthetic patient notes for global biomedical research. Materials and Methods The study used de-identified and re-identified versions of the MIMIC III dataset with GPT-3.5, GPT-4, and Mistral 7B to generate synthetic clinical notes. Text generation employed templates and keyword extraction for contextually relevant notes, with One-shot generation for comparison. Privacy was assessed by analyzing protected health information (PHI) occurrence and co-occurrence, while utility was evaluated by training an ICD-9 coder using synthetic notes. Text quality was measured using ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and cosine similarity metrics to compare synthetic notes with source notes for semantic similarity.

Results

The analysis of PHI occurrence and text utility via the ICD-9 coding task showed that the keyword-based method had low risk and good performance. One-shot generation exhibited the highest PHI exposure and PHI co-occurrence, particularly in geographic location and date categories. The Normalized One-shot method achieved the highest classification accuracy. Re-identified data consistently outperformed de-identified data.

Discussion

Privacy analysis revealed a critical balance between data utility and privacy protection, influencing future data use and sharing.

Conclusion

This study shows that keyword-based methods can create synthetic clinical notes that protect privacy while retaining data usability, potentially improving clinical data sharing. The use of dummy PHIs to counter privacy attacks may offer better utility and privacy than traditional de-identification.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingElectronic Health RecordsConfidentialityComputer SecurityHumansHealth Insurance Portability and Accountability ActUnited StatesLarge Language Models

Stichwörter der Autor:innen

confidentialitydata anonymizationlarge language modelsnatural language processingtext generation

Autor:innen und Institutionen

  • McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States.
  • Department of Computer Science, University of Manitoba, Winnipeg, Manitoba R3T 5V6, Canada.

ORCID-Profile: Yu-Chun Hsu, Xiaoqian Jiang

Förderung

  • NCI NIH HHS
  • NSERC Discovery Grants
  • National Science Foundation
  • NIA NIH HHS
  • Cancer Research
  • NCATS NIH HHS
  • NSERC
  • NLM NIH HHS

Zitieren

APA

Chuang, Y. S., Sarkar, A. R., Hsu, Y. C., Mohammed, N., & Jiang, X. (2025). Robust privacy amidst innovation with large language models through a critical assessment of the risks. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaf037

Vancouver

Chuang YS, Sarkar AR, Hsu YC, Mohammed N, Jiang X. Robust privacy amidst innovation with large language models through a critical assessment of the risks. Journal of the American Medical Informatics Association. 2025. doi:10.1093/jamia/ocaf037

BibTeX
@article{Chuang2025,
  author = {Chuang, Yao-Shun and Sarkar, Atiquer Rahman and Hsu, Yu-Chun and Mohammed, Noman and Jiang, Xiaoqian},
  title = {Robust privacy amidst innovation with large language models through a critical assessment of the risks},
  journal = {Journal of the American Medical Informatics Association},
  year = {2025},
  doi = {10.1093/jamia/ocaf037}
}
Kostenlos starten →