EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Exploring Prompt-Based Large Language Model (LLM) Approach for Medication Error-Related Named Entity Recognition in Medical Incident Reports

Mizue Ogi, Shin Ushiro, Zoie Shui-Yee Wong

Studies in Health Technology and Informatics · 2025

DOI: 10.3233/shti250938 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Mizue Ogi und 2 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Public Health, St.Luke's International University, Tokyo, Japan. Hauptthemen laut MEDLINE: Medication Errors, Natural Language Processing, Electronic Health Records. Die Arbeit wurde bisher 2-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Medication errors significantly challenge healthcare, necessitating innovative analytical methods. This study explored generative pre-trained language models (LLMs) for Named Entity Recognition (NER) in Japanese medical incident reports. We assessed four LLMs—Llama-3-ELYZA, BioMistral-7B, GPT-4.0 mini, and GPT-4.0—using a national open-source dataset, comparing their NER performance with a published annotated version of the data and offering a prompt-based framework approach to address the clinical NER problem. Although GPT-4.0 outperformed the others, it does not exceed the fine-tuned BERT model previously reported. Few-shot prompts achieved high accuracy for number-related entities (e.g., ‘Strength_rate’, F1-score: 0.951) matching the previous study, but clinically specific types underperformed due to language complexities. Despite these challenges, providing entity type definitions and a few examples improved GPT-4.0’s performance, highlighting LLMs’ potential without extensive training and the necessity of considering the linguistic challenges in the clinical NER problem.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Medication ErrorsNatural Language ProcessingElectronic Health RecordsHumansJapanLarge Language Models

Stichwörter der Autor:innen

LLMNERmedical incident reportsmedication errors

Autor:innen und Institutionen

  • Graduate School of Public Health, St.Luke's International University, Tokyo, Japan.
  • Division of Patient Safety, Kyushu University Hospital, Fukuoka, Japan.
  • Japan Council for Quality Health Care (JQ), Tokyo, Japan.

Zitieren

APA

Ogi, M., Ushiro, S., & Wong, Z. S. (2025). Exploring Prompt-Based Large Language Model (LLM) Approach for Medication Error-Related Named Entity Recognition in Medical Incident Reports. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250938

Vancouver

Ogi M, Ushiro S, Wong ZS. Exploring Prompt-Based Large Language Model (LLM) Approach for Medication Error-Related Named Entity Recognition in Medical Incident Reports. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250938

BibTeX
@article{Ogi2025,
  author = {Ogi, Mizue and Ushiro, Shin and Wong, Zoie Shui-Yee},
  title = {Exploring Prompt-Based Large Language Model (LLM) Approach for Medication Error-Related Named Entity Recognition in Medical Incident Reports},
  journal = {Studies in Health Technology and Informatics},
  year = {2025},
  doi = {10.3233/shti250938}
}
Kostenlos starten →