Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text
Studies in Health Technology and Informatics · 2025
DOI: 10.3233/shti251374 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Richard Zowalla und 1 weiteren Autor:innen. Beteiligte Einrichtung: Faculty for Informatics, Heilbronn University, Heilbronn, Germany. Hauptthemen laut MEDLINE: Natural Language Processing, Electronic Health Records, Patient Discharge Summaries. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Introduction
Detecting negations in clinical text is crucial for accurate documentation and decision-making.
Methods
This study assesses open-source Large Language Models (LLMs) for detecting negations in German clinical discharge letters, comparing them to the rule-based approach (GeNeg) and human annotations.
Results
While Llama 3.3 and Deepseek-R1 (70B) showed slight accuracy improvements, their high computational costs limit practicality compared to GeNeg. Llama 3.3 achieved the highest accuracy (.9670) and F1-score (.9620), outperforming all other models and slightly exceeding GeNeg in accuracy and F1-score. However, it required significantly more computational time (5.9 sec/sent) when compared to GeNeg’s processing time (.005 sec/sent).
Conclusion
The study results suggest hybrid approaches combining rule-based efficiency paired with LLMs’ linguistic capabilities. In addition, future work should therefore optimize prompts and integrate LLMs with traditional methods to balance accuracy and efficiency.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty for Informatics, Heilbronn University, Heilbronn, Germany.
- Fraunhofer Institute for Industrial Engineering IAO, Stuttgart, Germany.
Zitieren
APA
Zowalla, R., & Wiesner, M. (2025). Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti251374
Vancouver
Zowalla R, Wiesner M. Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti251374
BibTeX
@article{Zowalla2025,
author = {Zowalla, Richard and Wiesner, Martin},
title = {Prompting Is All You Need – Until It Isn’t: Exploring the Limits of LLMs for Negation Detection in German Clinical Text},
journal = {Studies in Health Technology and Informatics},
year = {2025},
doi = {10.3233/shti251374}
}
EUVIMED