EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Case-matched retrieval improves textual alignment of LLM-generated radiology impressions

Vera Sorin iD, Jeremy D Collins, Lewis D Hahn, Alex K Bratt, Eyal Klang, Panagiotis Korfiatis iD

PLOS One · 31.07.2026 · 21 (7) · e0354688

DOI: 10.1371/journal.pone.0354688 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 31.07.2026. Verfasst von Vera Sorin und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Mayo Clinic College of Medicine and Science, Mayo Clinic, Rochester, Minnesota, United States of America. Hauptthemen laut MEDLINE: Information Storage and Retrieval. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Radiology impressions guide clinical care. Large Language Models (LLMs)-drafted impressions can drift into generic, off-style text. Retrieval-augmented generation (RAG) enables context-aware few-shot prompting during inference.

Methods

This retrospective IRB-approved study included 11,998 CT pulmonary angiography (CTPA) reports. We built a retrieval bank from 11,399 reports and reserved 599 reports for testing. GPT-4o and LLaMA 3.1-70B generated impressions from the “findings” section using three setups: zero-shot, fixed random few-shot, and dynamic retrieval-selected few-shot (top-k semantic matches; k = 3/5/10). We ran temperatures 0, 0.7, 1. We scored outputs against the original impressions with ROUGE and BERTScore F1, report mean scores with 95% confidence intervals, and tested for statistical significance using Wilcoxon signed-rank test.

Results

Dynamic retrieval-based few-shot prompting outperformed zero-shot and fixed few-shot prompting across all configurations (all p < 0.05). The highest scores were observed at temperature 0 and k = 10. ROUGE-1 F1 increased to 0.44–0.47 for GPT-4o and 0.37–0.50 for LLaMA, versus 0.35–0.37 and 0.25–0.37, respectively, in zero-shot prompting. Lower temperature and larger k were associated with higher similarity scores.

Conclusions

Dynamic, case-matched retrieval improved alignment of LLM-generated CTPA impressions with reference impressions on automated text-similarity metrics. Scores remained moderate, and radiologists’ verification is still required before clinical deployment.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Information Storage and RetrievalLarge Language ModelsHumansRetrospective Studies

Autor:innen und Institutionen

  • Department of Radiology, Mayo Clinic College of Medicine and Science, Mayo Clinic, Rochester, Minnesota, United States of America.
  • The Windreich Department of Artificial Intelligence and Human Health, Mount Sinai Health System and the Icahn School of Medicine, Mount Sinai, New York, United States of America.

ORCID-Profile: Vera Sorin, Panagiotis Korfiatis

Zitieren

APA

Sorin, V., Collins, J. D., Hahn, L. D., Bratt, A. K., Klang, E., & Korfiatis, P. (2026). Case-matched retrieval improves textual alignment of LLM-generated radiology impressions. PLOS One, 21(7), e0354688. https://doi.org/10.1371/journal.pone.0354688

Vancouver

Sorin V, Collins JD, Hahn LD, Bratt AK, Klang E, Korfiatis P. Case-matched retrieval improves textual alignment of LLM-generated radiology impressions. PLOS One. 2026;21(7):e0354688. doi:10.1371/journal.pone.0354688

BibTeX
@article{Sorin2026,
  author = {Sorin, Vera and Collins, Jeremy D and Hahn, Lewis D and Bratt, Alex K and Klang, Eyal and Korfiatis, Panagiotis},
  title = {Case-matched retrieval improves textual alignment of LLM-generated radiology impressions},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {7},
  pages = {e0354688},
  doi = {10.1371/journal.pone.0354688}
}
Kostenlos starten →