RadOncRAG: A Novel Retrieval-Augmented Generation Framework Improves Large Language Model Benchmark Performance in Radiation Oncology
JCO Clinical Cancer Informatics · 2025
DOI: 10.1200/cci-25-00220 ↗Kurz gefasst
Fachartikel in der Zeitschrift JCO Clinical Cancer Informatics, 2025. Verfasst von Nikhil Gautam Thaker und 10 weiteren Autor:innen. Beteiligte Einrichtung: Capital Health, Pennington, NJ. Hauptthemen laut MEDLINE: Radiation Oncology, Benchmarking, Neoplasms, Information Storage and Retrieval. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 36 Literaturangaben.
Abstract (Originalsprache)
PURPOSE Large language models (LLMs) show promise in assisting knowledge-intensive fields such as oncology, where up-to-date information and multidisciplinary expertise are critical. Traditional LLMs risk hallucinations and reliance on static, possibly outdated data that lack domain-specific context. Retrieval-augmented generation (RAG) has emerged as a strategy to address these issues by incorporating domain-specific information from external knowledge repositories. METHODS We evaluated 15 LLMs, including Meta Llama-2/3, generative pretrained transformer (GPT)-3.5/4/4o variants, Claude-3, Gemini-2.0, and DeepSeek-R1. In a zero-shot workflow, each LLM answered 298 scorable questions from the 2021 American College of Radiology in-training examination. We implemented a RAG pipeline (Iridium Model) that transforms user prompts into vector embeddings, queries a specialized radiation oncology database, and merges relevant text with the original prompt to form an augmented query. We compared zero-shot versus RAG-augmented performance. RESULTS Larger-parameter LLMs had higher zero-shot accuracy, with six models outscoring graduating residents ( P < .01). Top scorers were reasoning models GPT-4o1, o3-mini, and DeepSeek-R1, which achieved 91.6%, 86.6%, and 91.6% without RAG, respectively. Gemini-2.0 improved 6.7% (to 79.2%), Llama-3-70b 8.4% (to 75.8%), and GPT-4o 5.7% (to 85.6%) with RAG. Top scoring reasoning models surpassed graduating resident averages by 17.7%-20% ( P < .01), but had no improvement or detriment with RAG. Domain-specific gains occurred in clinical, biology, and physics. Majority voting boosted aggregate accuracy when individual model performance exceeded 50%. RAG workflows and reasoning models incurred higher computational costs. CONCLUSION Radiation-oncology–specific retrieval-augmented generation pipeline enhances nonreasoning LLM performance in radiation oncology by integrating domain-specific evidence, whereas it does not improve performance of reasoning models. These findings demonstrate that RAG can elevate clinical decision support by enabling simpler, cost-effective nonreasoning models to tackle complex tasks through retrieval capabilities—an efficient alternative to extensive model training that also yields citable, evidence-based explanations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Capital Health, Pennington, NJ.
- Bayta Systems, Newtown, PA.
- Thomas Jefferson, Philadelphia, PA.
- St Luke's University Health Network, Easton, PA.
- Massive Bio, New York, NY.
- Atrium Health Wake Forest Baptist, Winston Salem, NC.
ORCID-Profile: Nikhil Gautam Thaker, Navid Redjal, Adam Dicker, Arturo Loaiza-Bonilla, Trevor Royce, Vivek Subbiah
Zitieren
APA
Thaker, N. G., Redjal, N., Dicker, A., Loaiza-Bonilla, A., Royce, T., Subbiah, V., Deendyal, V., Gabriel, J. R., Shetty, N., Choudhri, A., & Thaker, G. H. (2025). RadOncRAG: A Novel Retrieval-Augmented Generation Framework Improves Large Language Model Benchmark Performance in Radiation Oncology. JCO Clinical Cancer Informatics. https://doi.org/10.1200/cci-25-00220
Vancouver
Thaker NG, Redjal N, Dicker A, Loaiza-Bonilla A, Royce T, Subbiah V, et al. RadOncRAG: A Novel Retrieval-Augmented Generation Framework Improves Large Language Model Benchmark Performance in Radiation Oncology. JCO Clinical Cancer Informatics. 2025. doi:10.1200/cci-25-00220
BibTeX
@article{Thaker2025,
author = {Thaker, Nikhil Gautam and Redjal, Navid and Dicker, Adam and Loaiza-Bonilla, Arturo and Royce, Trevor and Subbiah, Vivek and Deendyal, Vikash and Gabriel, Jonathan R and Shetty, Neena and Choudhri, Ajay and Thaker, Gautam H},
title = {RadOncRAG: A Novel Retrieval-Augmented Generation Framework Improves Large Language Model Benchmark Performance in Radiation Oncology},
journal = {JCO Clinical Cancer Informatics},
year = {2025},
doi = {10.1200/cci-25-00220}
}
EUVIMED