Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation
JMIR Medical Informatics · 2026
DOI: 10.2196/87368 ↗At a glance
Journal article in JMIR Medical Informatics, 2026. Written by Songsoo Kim and 6 co-authors. Institution: Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea. Major MEDLINE topics: Large Language Models, Radiology, Diagnostic Errors, Radiology Information Systems. Cited 1 times so far (Crossref); 33 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Large language model (LLM) proofreaders for radiology reports generate many false positives (FPs) due to the low prevalence of errors.
Objective
This study aimed to determine whether an optimized LLM framework could improve both precision and cost-efficiency without compromising error detection capability.
Methods
In this retrospective study, 1000 radiology reports (radiography, ultrasonography, computed tomography, and magnetic resonance imaging; 250 each) were sampled from the Medical Information Mart for Intensive Care III database. Two public chest radiography corpora (CheXpert and Open-i) served as external test sets. Three LLM frameworks were evaluated: single-prompt detector (framework 1); report extractor plus single-prompt detector (framework 2); and extractor, detector, and FP verifier (framework 3). Precision for each framework was assessed using positive predictive value (PPV) and detected errors per 1000 reports. Overall efficiency was estimated using model inference costs and reviewer labor costs.
Results
PPV increased from 0.063 (95% CI 0.036‐0.101) in framework 1 to 0.079 (95% CI 0.049‐0.118) in framework 2 and 0.159 (95% CI 0.090‐0.252) in framework 3 ( P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea.
- Department of Radiology, Gangnam Severance Hospital, Seoul, Republic of Korea.
- Department of Internal Medicine, Gangnam Severance Hospital, Seoul, Republic of Korea.
- Department of Neurology, Severance Hospital, Seoul, Republic of Korea.
- Department of Surgery, Samsung Medical Center, Seoul, Republic of Korea.
- Department of Obstetrics and Gynecology, Kangbuk Samsung Hospital, Seoul, Republic of Korea.
ORCID profiles: Songsoo Kim, Seungtae Lee, See Young Lee, Joonho Kim, Keechan Kan, Hyunji Lee, Dukyong Yoon
Cite
APA
Kim, S., Lee, S., Lee, S. Y., Kim, J., Kan, K., Lee, H., & Yoon, D. (2026). Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation. JMIR Medical Informatics. https://doi.org/10.2196/87368
Vancouver
Kim S, Lee S, Lee SY, Kim J, Kan K, Lee H, et al. Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation. JMIR Medical Informatics. 2026. doi:10.2196/87368
BibTeX
@article{Kim2026,
author = {Kim, Songsoo and Lee, Seungtae and Lee, See Young and Kim, Joonho and Kan, Keechan and Lee, Hyunji and Yoon, Dukyong},
title = {Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation},
journal = {JMIR Medical Informatics},
year = {2026},
doi = {10.2196/87368}
}
EUVIMED