A Transformer-Based Pipeline for German Clinical Document De-Identification
Applied Clinical Informatics · 2025-01 · 16 (01) · 031-043
DOI: 10.1055/a-2424-1989 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Clinical Informatics, online erschienen am 08.01.2025. Verfasst von Kamyar Arzideh und 6 weiteren Autor:innen. Beteiligte Einrichtung: Central IT Department, Data Integration Center, University Hospital Essen, Essen, Germany. Hauptthemen laut MEDLINE: Documentation. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Deutsches Zentrum für Luft- und Raumfahrt.
Abstract (Originalsprache)
Objective
Commercially available large language models such as Chat Generative Pre-Trained Transformer (ChatGPT) cannot be applied to real patient data for data protection reasons. At the same time, de-identification of clinical unstructured data is a tedious and time-consuming task when done manually. Since transformer models can efficiently process and analyze large amounts of text data, our study aims to explore the impact of a large training dataset on the performance of this task.
Methods
We utilized a substantial dataset of 10,240 German hospital documents from 1,130 patients, created as part of the investigating hospital's routine documentation, as training data. Our approach involved fine-tuning and training an ensemble of two transformer-based language models simultaneously to identify sensitive data within our documents. Annotation Guidelines with specific annotation categories and types were created for annotator training.
Results
Performance evaluation on a test dataset of 100 manually annotated documents revealed that our fine-tuned German ELECTRA (gELECTRA) model achieved an F1 macro average score of 0.95, surpassing human annotators who scored 0.93.
Conclusion
We trained and evaluated transformer models to detect sensitive information in German real-world pathology reports and progress notes. By defining an annotation scheme tailored to the documents of the investigating hospital and creating annotation guidelines for staff training, a further experimental study was conducted to compare the models with humans. These results showed that the best-performing model achieved better overall results than two experienced annotators who manually labeled 100 clinical documents.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Central IT Department, Data Integration Center, University Hospital Essen, Essen, Germany.
- Institute for Artificial Intelligence in Medicine, University Hospital Essen, Essen, Germany.
- Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany.
- Department of Computer Science, University of Applied Sciences and Arts Dortmund, Dortmund, Germany.
- Institute for Medical Informatics, Biometry and Epidemiology, University Hospital Essen, Essen, Germany.
Förderung
- Deutsches Zentrum für Luft- und Raumfahrt
Zitieren
APA
Arzideh, K., Baldini, G., Winnekens, P., Friedrich, C. M., Nensa, F., Idrissi-Yaghir, A., & Hosch, R. (2025). A Transformer-Based Pipeline for German Clinical Document De-Identification. Applied Clinical Informatics, 16(01), 031-043. https://doi.org/10.1055/a-2424-1989
Vancouver
Arzideh K, Baldini G, Winnekens P, Friedrich CM, Nensa F, Idrissi-Yaghir A, et al. A Transformer-Based Pipeline for German Clinical Document De-Identification. Applied Clinical Informatics. 2025;16(01):031-043. doi:10.1055/a-2424-1989
BibTeX
@article{Arzideh2025,
author = {Arzideh, Kamyar and Baldini, Giulia and Winnekens, Philipp and Friedrich, Christoph M and Nensa, Felix and Idrissi-Yaghir, Ahmad and Hosch, René},
title = {A Transformer-Based Pipeline for German Clinical Document De-Identification},
journal = {Applied Clinical Informatics},
year = {2025},
volume = {16},
number = {01},
pages = {031--043},
doi = {10.1055/a-2424-1989}
}
EUVIMED