Large language models with retrieval-augmented generation enhance expert modelling of Bayesian network for clinical decision support
International Journal of Computer Assisted Radiology and Surgery · 03.11.2025 · 21 (2) · 211-222
DOI: 10.1007/s11548-025-03524-9 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, online erschienen am 03.11.2025. Verfasst von Mario A Cypko und 6 weiteren Autor:innen. Beteiligte Einrichtung: Hahn-Schickard-Gesellschaft für angewandte Forschung e.V., 79110, Freiburg, Germany. cypko@informatik.uni-freiburg.de. Hauptthemen laut MEDLINE: Decision Support Systems, Clinical. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Albert-Ludwigs-Universität Freiburg im Breisgau.
Abstract (Originalsprache)
Purpose
Bayesian networks (BNs) are valuable for clinical decision support due to their transparency and interpretability. However, BN modelling requires considerable manual effort. This study explores how integrating large language models (LLMs) with retrieval-augmented generation (RAG) can improve BN modelling by increasing efficiency, reducing cognitive workload, and ensuring accuracy.
Methods
We developed a web-based BN modelling service that integrates an LLM-RAG pipeline. A fine-tuned GTE-Large embedding model was employed for knowledge retrieval, optimised through recursive chunking and query expansion. To ensure accurate BN suggestions, we defined a causal structure for medical idioms by unifying existing BN frameworks. GPT-4 and Mixtral 8x7B were used to handle complex data interpretation and to generate modelling suggestions, respectively. A user study with four clinicians assessed usability, retrieval accuracy, and cognitive workload using NASA-TLX. The study demonstrated the system’s potential for efficient and clinically relevant BN modelling.
Results
The RAG pipeline improved retrieval accuracy and answer relevance. Recursive chunking with the fine-tuned embedding model GTE-Large achieved the highest retrieval accuracy score (0.9). Query expansion and Hyde optimisation enhanced retrieval accuracy for semantic chunking (0.75 to 0.85). Responses maintained high faithfulness ( $$\ge $$ ≥ 0.9). However, the LLM occasionally failed to adhere to predefined causal structures and medical idioms. All clinicians, regardless of BN experience, created comprehensive models within one hour. Experienced clinicians produced more complex models, but occasionally introduced causality errors, while less experienced users adhered more accurately to predefined structures. The tool reduced cognitive workload (2/7 NASA-TLX) and was described as intuitive, although workflow interruptions and minor technical issues highlighted areas for improvement.
Conclusion
Integrating LLM-RAG into BN modelling enhances efficiency and accuracy. Future work may focus on automated preprocessing, refinements of the user interface, and extending the RAG pipeline with validation steps and external biomedical sources. Generative AI holds promise for expert-driven knowledge modelling.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Hahn-Schickard-Gesellschaft für angewandte Forschung e.V., 79110, Freiburg, Germany. cypko@informatik.uni-freiburg.de.
- Intelligent Embedded Systems Lab, University of Freiburg, 79110, Freiburg, Germany. cypko@informatik.uni-freiburg.de.
- Hahn-Schickard-Gesellschaft für angewandte Forschung e.V., 79110, Freiburg, Germany.
- Intelligent Embedded Systems Lab, University of Freiburg, 79110, Freiburg, Germany.
- Department of Radiology, Staten Island University Hospital - Northwell Health, NY 10305, Staten Island, USA.
- Department of Otolaryngology, Head and Neck Surgery, University Hospital Leipzig, 04103, Leipzig, Germany.
Förderung
- Albert-Ludwigs-Universität Freiburg im Breisgau
Zitieren
APA
Cypko, M. A., Salim, M. A., Kumar, A., Berliner, L., Dietz, A., Stoehr, M., & Amft, O. (2025). Large language models with retrieval-augmented generation enhance expert modelling of Bayesian network for clinical decision support. International Journal of Computer Assisted Radiology and Surgery, 21(2), 211-222. https://doi.org/10.1007/s11548-025-03524-9
Vancouver
Cypko MA, Salim MA, Kumar A, Berliner L, Dietz A, Stoehr M, et al. Large language models with retrieval-augmented generation enhance expert modelling of Bayesian network for clinical decision support. International Journal of Computer Assisted Radiology and Surgery. 2025;21(2):211-222. doi:10.1007/s11548-025-03524-9
BibTeX
@article{Cypko2025,
author = {Cypko, Mario A and Salim, Muhammad Agus and Kumar, Aditya and Berliner, Leonard and Dietz, Andreas and Stoehr, Matthaeus and Amft, Oliver},
title = {Large language models with retrieval-augmented generation enhance expert modelling of Bayesian network for clinical decision support},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2025},
volume = {21},
number = {2},
pages = {211--222},
doi = {10.1007/s11548-025-03524-9}
}
EUVIMED