Large Language Model–Assisted Radiology Report Quality Assurance: Real-World Multicenter Evaluation
Journal of Medical Internet Research · 28.10.2026 · 28 · e106250-e106250
DOI: 10.2196/106250 ↗Kurz gefasst
Multizentrische Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 28.10.2026. Verfasst von Lujing Wang und 16 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, The Second Affiliated Hospital of Kunming Medical University, Kunming, Yunnan, 650101, China, 86 0871-63402918. Hauptthemen laut MEDLINE: Large Language Models, Quality Assurance, Health Care, Radiology. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Radiology reports serve as the definitive clinical record of imaging diagnoses, with their accuracy being directly critical to patient safety. However, amid escalating workloads, manual review remains inadequate for capturing subtle errors in reports on a large scale. Although large language models (LLMs) have demonstrated considerable promise in medical text processing, previous studies have predominantly been confined to radiology report text from a single imaging modality or datasets with synthetic errors.
Objective
This study aimed to evaluate the feasibility, accuracy, and generalizability of general-purpose LLMs for quality assurance of radiology report text from multiple imaging modalities (x-ray, computed tomography [CT], and magnetic resonance imaging [MRI]), multi-anatomical regions, and multicenter clinical settings, with head-to-head comparisons conducted against radiologists of varying seniority.
Methods
Radiology report texts from 3 aforementioned imaging modalities and 5 anatomical regions (head, chest, abdomen, spine, and knee) were retrospectively collected from center 1 (March 2025-May 2025) and reviewed using a clinician-annotated 5-category error taxonomy (type 1: interpretive; type 2: factual). Five LLMs (Qwen3, GPT-5.4 Pro, Gemini 3.1 Pro, Llama 4, and DeepSeek-V3.1) were benchmarked on 10,000 randomly selected reports, and the best-performing model was evaluated on the full cohort. A blinded head-to-head comparison with 3 radiologists (3-10 years of experience) used a modality-stratified, error-enriched challenge set. The radiologists subsequently rated the model-generated explanations using a 5-point Likert scale. Cross-center performance was evaluated in an independent external cohort. The same 2 senior radiologists labeled reports from both centers using the same protocol, resolving disagreements by consensus.
Results
In total, 4.6% (4431/95,582) of the reports in center 1 contained at least one confirmed text internal error. Qwen3, the selected model, achieved a report-level sensitivity, specificity, positive predictive value, and F 1 -score of 0.99, 0.99, 0.90, and 0.94, respectively, with high performance across reports from the 3 imaging modalities and multiple anatomical regions and better detection of factual than interpretive errors. In the error-enriched challenge set, its error instance exact category recall exceeded that of the radiologists (0.96 vs 0.45-0.67; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology, The Second Affiliated Hospital of Kunming Medical University, Kunming, Yunnan, 650101, China, 86 0871-63402918.
- Department of Radiology, The Sixth Affiliated Hospital of Kunming Medical University, Yuxi, Yunnan, China.
- School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China.
- Department of Radiology, School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom.
- Department of Radiology, National Center for Cardiovascular Diseases, Fuwai Hospital, Beijing, China.
Zitieren
APA
Wang, L., Zhang, Y., Peng, L., Liu, P., Wang, L., Cui, Y., Chen, Y., Li, X., Tao, Y., Gong, X., Zhao, X., Song, Y., Chen, J., Zhao, S., Wang, F., Ma, J., & Zhao, X. (2026). Large Language Model–Assisted Radiology Report Quality Assurance: Real-World Multicenter Evaluation. Journal of Medical Internet Research, 28, e106250-e106250. https://doi.org/10.2196/106250
Vancouver
Wang L, Zhang Y, Peng L, Liu P, Wang L, Cui Y, et al. Large Language Model–Assisted Radiology Report Quality Assurance: Real-World Multicenter Evaluation. Journal of Medical Internet Research. 2026;28:e106250-e106250. doi:10.2196/106250
BibTeX
@article{Wang2026,
author = {Wang, Lujing and Zhang, Yushuang and Peng, Liang and Liu, Pei and Wang, Li and Cui, Yu and Chen, Yixin and Li, Xingyu and Tao, Yitian and Gong, Xiuxian and Zhao, Xiaoying and Song, Yujiao and Chen, Jian and Zhao, Shihua and Wang, Fuying and Ma, Jian and Zhao, Xinxiang},
title = {Large Language Model–Assisted Radiology Report Quality Assurance: Real-World Multicenter Evaluation},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e106250--e106250},
doi = {10.2196/106250}
}
EUVIMED