EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text

Richard Zowalla, Martin Wiesner

Studies in Health Technology and Informatics · 2025

DOI: 10.3233/shti251374 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Richard Zowalla und 1 weiteren Autor:innen. Beteiligte Einrichtung: Faculty for Informatics, Heilbronn University, Heilbronn, Germany. Hauptthemen laut MEDLINE: Natural Language Processing, Electronic Health Records, Patient Discharge Summaries. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Detecting negations in clinical text is crucial for accurate documentation and decision-making.

Methods

This study assesses open-source Large Language Models (LLMs) for detecting negations in German clinical discharge letters, comparing them to the rule-based approach (GeNeg) and human annotations.

Results

While Llama 3.3 and Deepseek-R1 (70B) showed slight accuracy improvements, their high computational costs limit practicality compared to GeNeg. Llama 3.3 achieved the highest accuracy (.9670) and F1-score (.9620), outperforming all other models and slightly exceeding GeNeg in accuracy and F1-score. However, it required significantly more computational time (5.9 sec/sent) when compared to GeNeg’s processing time (.005 sec/sent).

Conclusion

The study results suggest hybrid approaches combining rule-based efficiency paired with LLMs’ linguistic capabilities. In addition, future work should therefore optimize prompts and integrate LLMs with traditional methods to balance accuracy and efficiency.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingElectronic Health RecordsPatient Discharge SummariesGermanyHumans

Stichwörter der Autor:innen

Artificial IntelligenceLarge Language ModelsLinguisticsNatural Language Processing

Autor:innen und Institutionen

  • Faculty for Informatics, Heilbronn University, Heilbronn, Germany.
  • Fraunhofer Institute for Industrial Engineering IAO, Stuttgart, Germany.

Zitieren

APA

Zowalla, R., & Wiesner, M. (2025). Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti251374

Vancouver

Zowalla R, Wiesner M. Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti251374

BibTeX
@article{Zowalla2025,
  author = {Zowalla, Richard and Wiesner, Martin},
  title = {Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text},
  journal = {Studies in Health Technology and Informatics},
  year = {2025},
  doi = {10.3233/shti251374}
}
Kostenlos starten →