EUVIMEDEUVIMED entdecken
Vergleichsstudie

Automated anonymization of radiology reports: comparison of publicly available natural language processing and large language models

Marcel C Langenbach, Borek Foldyna, Ibrahim Hadzic, Isabel L Langenbach, Vineet K Raghu, Michael T Lu, Tomas G Neilan, Julius C Heemelaar

European Radiology · 2024

DOI: 10.1007/s00330-024-11148-x ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift European Radiology, 2024. Verfasst von Marcel C Langenbach und 7 weiteren Autor:innen. Beteiligte Einrichtung: Cardiovascular Imaging Research Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA. mlangenbach@mgh.harvard.edu. Hauptthemen laut MEDLINE: Natural Language Processing, Data Anonymization, Radiology Information Systems. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 20 Literaturangaben. Gefördert von: Deutsche Forschungsgemeinschaft.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

PURPOSE: Medical reports, governed by HIPAA regulations, contain personal health information (PHI), restricting secondary data use. Utilizing natural language processing (NLP) and large language models (LLM), we sought to employ publicly available methods to automatically anonymize PHI in free-text radiology reports. MATERIALS AND METHODS: We compared two publicly available rule-based NLP models (spaCy; NLPac, accuracy-optimized; NLPsp, speed-optimized; iteratively improved on 400 free-text CT-reports (test set)) and one offline LLM approach (LLM-model, LLaMa-2, Meta-AI) for PHI-anonymization. The three models were tested on 100 randomly selected chest CT reports. Two investigators assessed the anonymization of occurring PHI entities and whether clinical information was removed. Subsequently, precision, recall, and F1 scores were calculated. RESULTS: NLPac and NLPsp successfully removed all instances of dates (n = 333), medical record numbers (MRN) (n = 6), and accession numbers (ACC) (n = 92). The LLM model removed all MRNs, 96% of ACCs, and 32% of dates. NLPac was most consistent with a perfect F1-score of 1.00, followed by NLPsp with lower precision (0.86) and F1-score (0.92) for dates. The LLM model had perfect precision for MRNs, ACCs, and dates but the lowest recall for ACC (0.96) and dates (0.52), corresponding F1 scores of 0.98 and 0.68, respectively. Names were removed completely or majorly (i.e., one first or family name non-anonymized) in 100% (NLPac), 72% (NLPsp), and 90% (LLM-model). Importantly, NLPac and NLPsp did not remove medical information, while the LLM model did in 10% (n = 10). CONCLUSION: Pre-trained NLP models can effectively anonymize free-text radiology reports, while anonymization with the LLM model is more prone to deleting medical information. KEY POINTS: Question This study compares NLP and locally hosted LLM techniques to ensure PHI anonymization without losing clinical information. Findings Pre-trained NLP models effectively anonymized radiology reports without removing clinical data, while a locally hosted LLM was less reliable, risking the loss of important information. Clinical relevance Fast, reliable, automated anonymization of PHI from radiology reports enables HIPAA-compliant secondary use, facilitating advanced applications like LLM-driven radiology analysis while ensuring ethical handling of sensitive patient data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingData AnonymizationRadiology Information SystemsHumansElectronic Health RecordsLarge Language Models

Stichwörter der Autor:innen

Data anonymizationDiagnostic imagingElectronic health recordsMachine learningNatural language processing

Autor:innen und Institutionen

  • Cardiovascular Imaging Research Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA. mlangenbach@mgh.harvard.edu.
  • Institute for Diagnostic and Interventional Radiology, University Hospital Cologne, Cologne, Germany. mlangenbach@mgh.harvard.edu.
  • Cardiovascular Imaging Research Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
  • Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, The Netherlands.
  • Institute for Diagnostic and Interventional Radiology, University Hospital Cologne, Cologne, Germany.
  • Department of Cardiology, Leiden University Medical Center, Leiden, The Netherlands.

Förderung

  • Deutsche Forschungsgemeinschaft

Zitieren

APA

Langenbach, M. C., Foldyna, B., Hadzic, I., Langenbach, I. L., Raghu, V. K., Lu, M. T., Neilan, T. G., & Heemelaar, J. C. (2024). Automated anonymization of radiology reports: comparison of publicly available natural language processing and large language models. European Radiology. https://doi.org/10.1007/s00330-024-11148-x

Vancouver

Langenbach MC, Foldyna B, Hadzic I, Langenbach IL, Raghu VK, Lu MT, et al. Automated anonymization of radiology reports: comparison of publicly available natural language processing and large language models. European Radiology. 2024. doi:10.1007/s00330-024-11148-x

BibTeX
@article{Langenbach2024,
  author = {Langenbach, Marcel C and Foldyna, Borek and Hadzic, Ibrahim and Langenbach, Isabel L and Raghu, Vineet K and Lu, Michael T and Neilan, Tomas G and Heemelaar, Julius C},
  title = {Automated anonymization of radiology reports: comparison of publicly available natural language processing and large language models},
  journal = {European Radiology},
  year = {2024},
  doi = {10.1007/s00330-024-11148-x}
}
Kostenlos starten →