EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering

Zafer Akcali iD, Hazal Selvi Cubuk iD, Arzu Oguz iD, Murat Kocak iD, Aydan Farzaliyeva, Fatih Guven, Mehmet Nezir Ramazanoglu iD, Efe Hasdemir, Ozden Altundag, Ahmet Muhtesem Agildere

Bioengineering · 10.02.2025 · 12 (2) · 168

DOI: 10.3390/bioengineering12020168 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioengineering, online erschienen am 10.02.2025. Verfasst von Zafer Akcali und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Informatics, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Baskent University Research Fund.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Named entity recognition (NER) offers a powerful method for automatically extracting key clinical information from text, but current models often lack sufficient support for non-English languages. Materials and Methods: This study investigated a prompt-based NER approach using Google’s Gemini 1.5 Pro, a large language model (LLM) with a 1.5-million-token context window. We focused on extracting important clinical entities from Turkish mammography reports, a language with limited available natural language processing (NLP) tools. Our method employed many-shot learning, incorporating 165 examples within a 26,000-token prompt derived from 75 initial reports. We tested the model on a separate set of 85 unannotated reports, concentrating on five key entities: anatomy (ANAT), impression (IMP), observation presence (OBS-P), absence (OBS-A), and uncertainty (OBS-U).

Results

Our approach achieved high accuracy, with a macro-averaged F1 score of 0.99 for relaxed match and 0.84 for exact match. In relaxed matching, the model achieved F1 scores of 0.99 for ANAT, 0.99 for IMP, 1.00 for OBS-P, 1.00 for OBS-A, and 0.99 for OBS-U. For exact match, the F1 scores were 0.88 for ANAT, 0.79 for IMP, 0.78 for OBS-P, 0.94 for OBS-A, and 0.82 for OBS-U.

Discussion

These results indicate that a many-shot prompt engineering approach with large language models provides an effective way to automate clinical information extraction for languages where NLP resources are less developed, and as reported in the literature, generally outperforms zero-shot, five-shot, and other few-shot methods.

Conclusion

This approach has the potential to significantly improve clinical workflows and research efforts in multilingual healthcare environments.

Schlagwörter

Stichwörter der Autor:innen

clinicomicsmachine learningmammographynamed entity recognition (NER)natural language processing (NLP)prompt engineeringradiology reports

Autor:innen und Institutionen

  • Department of Medical Informatics, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye.
  • Division of Medical Oncology, Department of Internal Medicine, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye.
  • Department of Radiology, Faculty of Medicine, Baskent University, Ankara 06790, Türkiye.

ORCID-Profile: Zafer Akcali, Hazal Selvi Cubuk, Arzu Oguz, Murat Kocak, Mehmet Nezir Ramazanoglu

Förderung

  • Baskent University Research Fund

Zitieren

APA

Akcali, Z., Cubuk, H. S., Oguz, A., Kocak, M., Farzaliyeva, A., Guven, F., Ramazanoglu, M. N., Hasdemir, E., Altundag, O., & Agildere, A. M. (2025). Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering. Bioengineering, 12(2), 168. https://doi.org/10.3390/bioengineering12020168

Vancouver

Akcali Z, Cubuk HS, Oguz A, Kocak M, Farzaliyeva A, Guven F, et al. Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering. Bioengineering. 2025;12(2):168. doi:10.3390/bioengineering12020168

BibTeX
@article{Akcali2025,
  author = {Akcali, Zafer and Cubuk, Hazal Selvi and Oguz, Arzu and Kocak, Murat and Farzaliyeva, Aydan and Guven, Fatih and Ramazanoglu, Mehmet Nezir and Hasdemir, Efe and Altundag, Ozden and Agildere, Ahmet Muhtesem},
  title = {Automated Extraction of Key Entities from Non-English Mammography Reports Using Named Entity Recognition with Prompt Engineering},
  journal = {Bioengineering},
  year = {2025},
  volume = {12},
  number = {2},
  pages = {168},
  doi = {10.3390/bioengineering12020168}
}
Kostenlos starten →