EUVIMEDEUVIMED entdecken
Open Access

Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation

Pengze Li, Anshum Patel, Sai Krishna Vallamchetla, Hayden Heninger, Het Contractor, Cui Tao, Joseph Cheung

Journal of the American Medical Informatics Association · 2026

DOI: 10.1093/jamia/ocag056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2026. Verfasst von Pengze Li und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL, United States. Hauptthemen laut MEDLINE: Information Storage and Retrieval, Sleep Medicine Specialty, Natural Language Processing. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Evaluate how RAG architecture, including corpus structure, retrieval strategy, and pipeline complexity, affects LLM-based medical problem solving and knowledge retrieval in sleep medicine. Materials and Methods We benchmarked four open-source LLMs (Llama-3-8B, Llama -3 -70B, Qwen 2.5-14B, and Qwen 2.5-235B) using a knowledge base of five sleep medicine textbooks. We compared performance across three dimensions: corpus structure (raw text vs table-of-contents aligned.), retrieval strategy (dense embedding vs hybrid sparse-dense), and pipeline complexity (baseline vs augmented). Evaluation metrics included board-style multiple choice question (MCQ) accuracy and clinical case vignette diagnostic ranking.

Results

RAG improved MCQ accuracy for all models. Llama-8B saw the largest gain of 10.6% (61.8% to 72.4%), while Qwen-235B reached 87.3%. In clinical cases, Llama-8B accuracy dropped by 7.1% when using raw text and dense retrieval due to context noise. This was corrected by using structured hybrid configurations. Hybrid retrieval consistently outperformed dense-only methods. Overall, structured corpora improved primary diagnosis accuracy by 6.1% on average, with Qwen-235B reaching a peak 10.2% increase.

Discussion

RAG effectiveness depends on the balance between model size and data structure. Large models handle uncurated text well, but smaller models are easily distracted by irrelevant data. Hybrid retrieval is necessary to maintain precision with specialized medical terms. A structured corpus paired with a baseline hybrid pipeline offers the best stability and speed for clinical use.

Conclusion

Rigorous data curation and hybrid retrieval are as essential as model scale for deploying safe, guideline-compliant AI in sleep medicine.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Information Storage and RetrievalSleep Medicine SpecialtyNatural Language ProcessingLarge Language ModelsHumansGenerative Artificial IntelligenceKnowledge BasesBenchmarking

Stichwörter der Autor:innen

Retrieval-Augmented Generationinformation retrievallarge language modelsnatural language processingsleep medicine

Autor:innen und Institutionen

  • Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL, United States.
  • Division of Pulmonary, Allergy and Sleep Medicine, Mayo Clinic, Jacksonville, FL, United States.
  • Department of Neurology, Mayo Clinic, Jacksonville, FL, United States.

Zitieren

APA

Li, P., Patel, A., Vallamchetla, S. K., Heninger, H., Contractor, H., Tao, C., & Cheung, J. (2026). Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocag056

Vancouver

Li P, Patel A, Vallamchetla SK, Heninger H, Contractor H, Tao C, et al. Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocag056

BibTeX
@article{Li2026,
  author = {Li, Pengze and Patel, Anshum and Vallamchetla, Sai Krishna and Heninger, Hayden and Contractor, Het and Tao, Cui and Cheung, Joseph},
  title = {Optimizing Retrieval-Augmented Generation (RAG) in clinical medicine: methods and performance evaluation},
  journal = {Journal of the American Medical Informatics Association},
  year = {2026},
  doi = {10.1093/jamia/ocag056}
}
Kostenlos starten →