Development of an LLM pipeline exceeding physician-documented cardiovascular risk scores under routine clinical conditions
European Heart Journal - Digital Health · 29.07.2026 · 7 (7)
DOI: 10.1093/ehjdh/ztag124 ↗Kurz gefasst
Fachartikel in der Zeitschrift European Heart Journal - Digital Health, online erschienen am 28.07.2026. Verfasst von Tobias Roeschl und 9 weiteren Autor:innen. Beteiligte Einrichtung: Deutsches Herzzentrum der Charité, Department of Cardiology, Angiology and Intensive Care Medicine, Charitéplatz 1, Berlin 10117, Germany. Die Arbeit nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: German Centre for Cardiovascular Research, Federal Ministry of Education and Research, Charité – Universitätsmedizin Berlin.
Abstract (Originalsprache)
Aims
Risk scores are essential to evidence-based cardiovascular care, but manual calculation is labour intensive and error prone. Large language models (LLMs) could automate this process, yet LLMs are limited by their propensity for calculation errors and factual hallucinations. Pipelines separating LLM-based data extraction from deterministic score computation may improve reliability and transparency. Methods and results We conducted a retrospective diagnostic study at a quaternary heart centre in Germany (January 2020 to July 2023). Patients with atrial fibrillation (n = 179) from an ablation registry and patients with severe aortic stenosis (n = 76) evaluated by a heart team were included. Six LLMs (GPT-5.2, Gemini 3.1 Pro, DeepSeek-R1, Qwen3, GPT-OSS 120B, and Kimi K2.5) were tested in standalone, retrieval-augmented generation (RAG), and pipeline configurations to compute HAS-BLED, CHA2DS2-VASc, and EuroSCORE II scores from routine clinical reports. Accuracy was assessed against expert-adjudicated ground truth using root mean squared error (RMSE) and Krippendorff’s α to evaluate numerical deviation and categorical agreement, respectively. Pipeline-generated scores showed substantially higher agreement with expert adjudication than standalone LLMs, LLMs with RAG, and treating physicians (mean Krippendorff’s α: 0.78 vs. 0.32 vs. 0.39 vs. 0.31) and lower deviation from ground truth (mean RMSE: 0.89 vs. 5.81 vs. 1.85 vs. 1.34).
Conclusion
Pipelines combining expert-curated knowledge injection, LLM-based clinical data extraction, and deterministic score calculation enable accurate and scalable cardiovascular risk score computation from unstructured real-world clinical data, outperforming physician-documented scores. Such pipelines could form the basis for clinical decision-support systems that automate routine risk assessment, reduce clinician workload, and promote more consistent evidence-based care.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Deutsches Herzzentrum der Charité, Department of Cardiology, Angiology and Intensive Care Medicine, Charitéplatz 1, Berlin 10117, Germany.
- Charité - Universitätsmedizin Berlin, corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charitéplatz 1, Berlin 10117, Germany.
- Charité - Universitätsmedizin Berlin, Institute for Artificial Intelligence in Medicine, Invalidenstraße 120, Berlin 10115, Germany.
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Charitéplatz 1, Berlin 10117, Germany.
- DZHK (German Centre for Cardiovascular Research), partner Site Berlin, Potsdamer Str. 58, 10785 Berlin, Germany.
- Deutsches Herzzentrum der Charité, Department of Cardiothoracic and Vascular Surgery, Augustenburger Platz 1, Berlin 13353, Germany.
ORCID-Profile: Tobias Roeschl, Gerhard Hindricks, Volkmar Falk, Ran Balicer, Radu Tanacli, Alexander Meyer
Förderung
- German Centre for Cardiovascular Research
- Federal Ministry of Education and Research
- Charité – Universitätsmedizin Berlin
- BIH Charité Digital Clinician Scientist Program
- Berlin Institute of Health
Zitieren
APA
Roeschl, T., Hoffmann, M., Unbehaun, A., Dreger, H., Hindricks, G., Falk, V., Balicer, R., Tanacli, R., Hohendanner, F., & Meyer, A. (2026). Development of an LLM pipeline exceeding physician-documented cardiovascular risk scores under routine clinical conditions. European Heart Journal - Digital Health, 7(7). https://doi.org/10.1093/ehjdh/ztag124
Vancouver
Roeschl T, Hoffmann M, Unbehaun A, Dreger H, Hindricks G, Falk V, et al. Development of an LLM pipeline exceeding physician-documented cardiovascular risk scores under routine clinical conditions. European Heart Journal - Digital Health. 2026;7(7). doi:10.1093/ehjdh/ztag124
BibTeX
@article{Roeschl2026,
author = {Roeschl, Tobias and Hoffmann, Marie and Unbehaun, Axel and Dreger, Henryk and Hindricks, Gerhard and Falk, Volkmar and Balicer, Ran and Tanacli, Radu and Hohendanner, Felix and Meyer, Alexander},
title = {Development of an LLM pipeline exceeding physician-documented cardiovascular risk scores under routine clinical conditions},
journal = {European Heart Journal - Digital Health},
year = {2026},
volume = {7},
number = {7},
doi = {10.1093/ehjdh/ztag124}
}
EUVIMED