Case-matched retrieval improves textual alignment of LLM-generated radiology impressions
PLOS One · 31.07.2026 · 21 (7) · e0354688
DOI: 10.1371/journal.pone.0354688 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 31.07.2026. Verfasst von Vera Sorin und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Mayo Clinic College of Medicine and Science, Mayo Clinic, Rochester, Minnesota, United States of America. Hauptthemen laut MEDLINE: Information Storage and Retrieval. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Radiology impressions guide clinical care. Large Language Models (LLMs)-drafted impressions can drift into generic, off-style text. Retrieval-augmented generation (RAG) enables context-aware few-shot prompting during inference.
Methods
This retrospective IRB-approved study included 11,998 CT pulmonary angiography (CTPA) reports. We built a retrieval bank from 11,399 reports and reserved 599 reports for testing. GPT-4o and LLaMA 3.1-70B generated impressions from the “findings” section using three setups: zero-shot, fixed random few-shot, and dynamic retrieval-selected few-shot (top-k semantic matches; k = 3/5/10). We ran temperatures 0, 0.7, 1. We scored outputs against the original impressions with ROUGE and BERTScore F1, report mean scores with 95% confidence intervals, and tested for statistical significance using Wilcoxon signed-rank test.
Results
Dynamic retrieval-based few-shot prompting outperformed zero-shot and fixed few-shot prompting across all configurations (all p < 0.05). The highest scores were observed at temperature 0 and k = 10. ROUGE-1 F1 increased to 0.44–0.47 for GPT-4o and 0.37–0.50 for LLaMA, versus 0.35–0.37 and 0.25–0.37, respectively, in zero-shot prompting. Lower temperature and larger k were associated with higher similarity scores.
Conclusions
Dynamic, case-matched retrieval improved alignment of LLM-generated CTPA impressions with reference impressions on automated text-similarity metrics. Scores remained moderate, and radiologists’ verification is still required before clinical deployment.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Radiology, Mayo Clinic College of Medicine and Science, Mayo Clinic, Rochester, Minnesota, United States of America.
- The Windreich Department of Artificial Intelligence and Human Health, Mount Sinai Health System and the Icahn School of Medicine, Mount Sinai, New York, United States of America.
ORCID-Profile: Vera Sorin, Panagiotis Korfiatis
Zitieren
APA
Sorin, V., Collins, J. D., Hahn, L. D., Bratt, A. K., Klang, E., & Korfiatis, P. (2026). Case-matched retrieval improves textual alignment of LLM-generated radiology impressions. PLOS One, 21(7), e0354688. https://doi.org/10.1371/journal.pone.0354688
Vancouver
Sorin V, Collins JD, Hahn LD, Bratt AK, Klang E, Korfiatis P. Case-matched retrieval improves textual alignment of LLM-generated radiology impressions. PLOS One. 2026;21(7):e0354688. doi:10.1371/journal.pone.0354688
BibTeX
@article{Sorin2026,
author = {Sorin, Vera and Collins, Jeremy D and Hahn, Lewis D and Bratt, Alex K and Klang, Eyal and Korfiatis, Panagiotis},
title = {Case-matched retrieval improves textual alignment of LLM-generated radiology impressions},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {7},
pages = {e0354688},
doi = {10.1371/journal.pone.0354688}
}
EUVIMED