EUVIMEDSkopri EUVIMED
Open access

Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation

Songsoo Kim iD, Seungtae Lee iD, See Young Lee iD, Joonho Kim iD, Keechan Kan iD, Hyunji Lee iD, Dukyong Yoon iD

JMIR Medical Informatics · 2026

DOI: 10.2196/87368 ↗

At a glance

Journal article in JMIR Medical Informatics, 2026. Written by Songsoo Kim and 6 co-authors. Institution: Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea. Major MEDLINE topics: Large Language Models, Radiology, Diagnostic Errors, Radiology Information Systems. Cited 1 times so far (Crossref); 33 references. The full text is freely available in PubMed Central.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

Large language model (LLM) proofreaders for radiology reports generate many false positives (FPs) due to the low prevalence of errors.

Objective

This study aimed to determine whether an optimized LLM framework could improve both precision and cost-efficiency without compromising error detection capability.

Methods

In this retrospective study, 1000 radiology reports (radiography, ultrasonography, computed tomography, and magnetic resonance imaging; 250 each) were sampled from the Medical Information Mart for Intensive Care III database. Two public chest radiography corpora (CheXpert and Open-i) served as external test sets. Three LLM frameworks were evaluated: single-prompt detector (framework 1); report extractor plus single-prompt detector (framework 2); and extractor, detector, and FP verifier (framework 3). Precision for each framework was assessed using positive predictive value (PPV) and detected errors per 1000 reports. Overall efficiency was estimated using model inference costs and reviewer labor costs.

Results

PPV increased from 0.063 (95% CI 0.036‐0.101) in framework 1 to 0.079 (95% CI 0.049‐0.118) in framework 2 and 0.159 (95% CI 0.090‐0.252) in framework 3 ( P

Subjects

MeSH terms (MEDLINE)

Large Language ModelsRadiologyDiagnostic ErrorsRadiology Information SystemsHumansRetrospective StudiesReproducibility of Results

Author keywords

error detectionhuman-in-the-looplarge language modelsquality assuranceradiology report

Authors and institutions

  • Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea.
  • Department of Radiology, Gangnam Severance Hospital, Seoul, Republic of Korea.
  • Department of Internal Medicine, Gangnam Severance Hospital, Seoul, Republic of Korea.
  • Department of Neurology, Severance Hospital, Seoul, Republic of Korea.
  • Department of Surgery, Samsung Medical Center, Seoul, Republic of Korea.
  • Department of Obstetrics and Gynecology, Kangbuk Samsung Hospital, Seoul, Republic of Korea.

ORCID profiles: Songsoo Kim, Seungtae Lee, See Young Lee, Joonho Kim, Keechan Kan, Hyunji Lee, Dukyong Yoon

Cite

APA

Kim, S., Lee, S., Lee, S. Y., Kim, J., Kan, K., Lee, H., & Yoon, D. (2026). Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation. JMIR Medical Informatics. https://doi.org/10.2196/87368

Vancouver

Kim S, Lee S, Lee SY, Kim J, Kan K, Lee H, et al. Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation. JMIR Medical Informatics. 2026. doi:10.2196/87368

BibTeX
@article{Kim2026,
  author = {Kim, Songsoo and Lee, Seungtae and Lee, See Young and Kim, Joonho and Kan, Keechan and Lee, Hyunji and Yoon, Dukyong},
  title = {Improving Radiology Report Error Detection Using a Multipass Large Language Model: Framework Development and Validation},
  journal = {JMIR Medical Informatics},
  year = {2026},
  doi = {10.2196/87368}
}
Start for free →