Clinical Code Mapping with LLM Tool Use: A Pilot for Automated Data Extraction of Medication and Diagnosis Information from Unstructured Clinical Notes
Studies in Health Technology and Informatics · 2026
DOI: 10.3233/shti260999 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2026. Verfasst von Til Spreuer und 2 weiteren Autor:innen. Beteiligte Einrichtung: Universities of Giessen and Marburg Lung Center, Department of Internal Medicine, Gießen, Germany. Hauptthemen laut MEDLINE: Large Language Models, Natural Language Processing, Electronic Health Records, Clinical Coding. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Introduction
Accurate clinical coding is fundamental to large-scale epidemiological studies, hospital billing, and the development of robust clinical decision support systems. Conventional methods for structured data extraction often rely on manual curation, which is prohibitively labor-intensive. Goal of this project is to determine whether current state-of-the-art open-weight LLM models are suitable for extraction of structured data from non-English (German) clinical notes.
Methods
We anonymized 35 German doctor’s notes of five patients from our hospital and developed one pipeline to extract and map medications and two for diagnoses. The latter compares a RAG based approach with an agentic AI. We ran these using three open-weight LLMs on a local GPU-PC.
Results
The F1 scores for diagnoses do not exceed 0.12. If we instead consider mapping to the broad category, then the F1 score increases to 0.18. For medications, the F1 score is as high as 0.78 and even 0.95 if we consider trivial name extraction only.
Discussion
For trivial name extraction of medications, every encountered mistake is explainable. Due to limitations in the nature of the task, it is infeasible to expect a perfect score of 1 in any of the coding scenarios. Further problems in LLM output and parsing are addressed.
Conclusion
LLMs excel at extraction. They are suitable for information extraction of medications from clinical notes for use in research databases. However, for a clinical setting where the treatment of patients would be dependent on LLM performance, the current state-of-the-art open weight models are not accurate enough.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Universities of Giessen and Marburg Lung Center, Department of Internal Medicine, Gießen, Germany.
- Cardio-Pulmonary Institute (CPI), Institute for Lung Health (ILH), Gießen, Germany.
- Institute of Medical Informatics, RWTH Aachen University Medical Faculty, Aachen, Germany.
Zitieren
APA
Spreuer, T., Günther, A., & Majeed, R. W. (2026). Clinical Code Mapping with LLM Tool Use: A Pilot for Automated Data Extraction of Medication and Diagnosis Information from Unstructured Clinical Notes. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti260999
Vancouver
Spreuer T, Günther A, Majeed RW. Clinical Code Mapping with LLM Tool Use: A Pilot for Automated Data Extraction of Medication and Diagnosis Information from Unstructured Clinical Notes. Studies in Health Technology and Informatics. 2026. doi:10.3233/shti260999
BibTeX
@article{Spreuer2026,
author = {Spreuer, Til and Günther, Andreas and Majeed, Raphael W},
title = {Clinical Code Mapping with LLM Tool Use: A Pilot for Automated Data Extraction of Medication and Diagnosis Information from Unstructured Clinical Notes},
journal = {Studies in Health Technology and Informatics},
year = {2026},
doi = {10.3233/shti260999}
}
EUVIMED