Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation
Journal of the American Medical Informatics Association · 2026
DOI: 10.1093/jamia/ocag056 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2026. Verfasst von Pengze Li und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL, United States. Hauptthemen laut MEDLINE: Information Storage and Retrieval, Sleep Medicine Specialty, Natural Language Processing. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Objective
Evaluate how RAG architecture, including corpus structure, retrieval strategy, and pipeline complexity, affects LLM-based medical problem solving and knowledge retrieval in sleep medicine. Materials and Methods We benchmarked four open-source LLMs (Llama-3-8B, Llama -3 -70B, Qwen 2.5-14B, and Qwen 2.5-235B) using a knowledge base of five sleep medicine textbooks. We compared performance across three dimensions: corpus structure (raw text vs table-of-contents aligned.), retrieval strategy (dense embedding vs hybrid sparse-dense), and pipeline complexity (baseline vs augmented). Evaluation metrics included board-style multiple choice question (MCQ) accuracy and clinical case vignette diagnostic ranking.
Results
RAG improved MCQ accuracy for all models. Llama-8B saw the largest gain of 10.6% (61.8% to 72.4%), while Qwen-235B reached 87.3%. In clinical cases, Llama-8B accuracy dropped by 7.1% when using raw text and dense retrieval due to context noise. This was corrected by using structured hybrid configurations. Hybrid retrieval consistently outperformed dense-only methods. Overall, structured corpora improved primary diagnosis accuracy by 6.1% on average, with Qwen-235B reaching a peak 10.2% increase.
Discussion
RAG effectiveness depends on the balance between model size and data structure. Large models handle uncurated text well, but smaller models are easily distracted by irrelevant data. Hybrid retrieval is necessary to maintain precision with specialized medical terms. A structured corpus paired with a baseline hybrid pipeline offers the best stability and speed for clinical use.
Conclusion
Rigorous data curation and hybrid retrieval are as essential as model scale for deploying safe, guideline-compliant AI in sleep medicine.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL, United States.
- Division of Pulmonary, Allergy and Sleep Medicine, Mayo Clinic, Jacksonville, FL, United States.
- Department of Neurology, Mayo Clinic, Jacksonville, FL, United States.
Zitieren
APA
Li, P., Patel, A., Vallamchetla, S. K., Heninger, H., Contractor, H., Tao, C., & Cheung, J. (2026). Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocag056
Vancouver
Li P, Patel A, Vallamchetla SK, Heninger H, Contractor H, Tao C, et al. Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocag056
BibTeX
@article{Li2026,
author = {Li, Pengze and Patel, Anshum and Vallamchetla, Sai Krishna and Heninger, Hayden and Contractor, Het and Tao, Cui and Cheung, Joseph},
title = {Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
doi = {10.1093/jamia/ocag056}
}
EUVIMED