Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering
Bioengineering · 10.02.2025 · 12 (2) · 168
DOI: 10.3390/bioengineering12020168 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioengineering, online erschienen am 10.02.2025. Verfasst von Zafer Akcali und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Informatics, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Baskent University Research Fund.
Abstract (Originalsprache)
Objective
Named entity recognition (NER) offers a powerful method for automatically extracting key clinical information from text, but current models often lack sufficient support for non-English languages. Materials and Methods: This study investigated a prompt-based NER approach using Google’s Gemini 1.5 Pro, a large language model (LLM) with a 1.5-million-token context window. We focused on extracting important clinical entities from Turkish mammography reports, a language with limited available natural language processing (NLP) tools. Our method employed many-shot learning, incorporating 165 examples within a 26,000-token prompt derived from 75 initial reports. We tested the model on a separate set of 85 unannotated reports, concentrating on five key entities: anatomy (ANAT), impression (IMP), observation presence (OBS-P), absence (OBS-A), and uncertainty (OBS-U).
Results
Our approach achieved high accuracy, with a macro-averaged F1 score of 0.99 for relaxed match and 0.84 for exact match. In relaxed matching, the model achieved F1 scores of 0.99 for ANAT, 0.99 for IMP, 1.00 for OBS-P, 1.00 for OBS-A, and 0.99 for OBS-U. For exact match, the F1 scores were 0.88 for ANAT, 0.79 for IMP, 0.78 for OBS-P, 0.94 for OBS-A, and 0.82 for OBS-U.
Discussion
These results indicate that a many-shot prompt engineering approach with large language models provides an effective way to automate clinical information extraction for languages where NLP resources are less developed, and as reported in the literature, generally outperforms zero-shot, five-shot, and other few-shot methods.
Conclusion
This approach has the potential to significantly improve clinical workflows and research efforts in multilingual healthcare environments.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medical Informatics, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye.
- Division of Medical Oncology, Department of Internal Medicine, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye.
- Department of Radiology, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye.
ORCID-Profile: Zafer Akcali, Hazal Selvi Cubuk, Arzu Oguz, Murat Kocak, Mehmet Nezir Ramazanoglu
Förderung
- Baskent University Research Fund
Zitieren
APA
Akcali, Z., Cubuk, H. S., Oguz, A., Kocak, M., Farzaliyeva, A., Guven, F., Ramazanoglu, M. N., Hasdemir, E., Altundag, O., & Agildere, A. M. (2025). Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering. Bioengineering, 12(2), 168. https://doi.org/10.3390/bioengineering12020168
Vancouver
Akcali Z, Cubuk HS, Oguz A, Kocak M, Farzaliyeva A, Guven F, et al. Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering. Bioengineering. 2025;12(2):168. doi:10.3390/bioengineering12020168
BibTeX
@article{Akcali2025,
author = {Akcali, Zafer and Cubuk, Hazal Selvi and Oguz, Arzu and Kocak, Murat and Farzaliyeva, Aydan and Guven, Fatih and Ramazanoglu, Mehmet Nezir and Hasdemir, Efe and Altundag, Ozden and Agildere, Ahmet Muhtesem},
title = {Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering},
journal = {Bioengineering},
year = {2025},
volume = {12},
number = {2},
pages = {168},
doi = {10.3390/bioengineering12020168}
}
EUVIMED