Human-in-the-Loop Large Language Model–Augmented Diagnostic Reasoning in Thoracic Imaging: Impact of Radiologic Expertise
American Journal of Roentgenology · 2026
DOI: 10.2214/ajr.26.34999 ↗Kurz gefasst
Fachartikel in der Zeitschrift American Journal of Roentgenology, 2026. Verfasst von Jiyoung Song und 14 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, 101 Daehakro, Jongno-gu, Seoul, 03080, Ko…. Hauptthemen laut MEDLINE: Large Language Models, Clinical Competence, Clinical Reasoning, Radiography, Thoracic. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 21 Literaturangaben.
Abstract (Originalsprache)
BACKGROUND. Technical and regulatory constraints limit application of large language models (LLMs) for augmenting diagnostic reasoning in radiology. Reader-mediated text-based workflows may provide a practical alternative. OBJECTIVE. The purpose of this study was to evaluate the impact on diagnostic performance of LLM assistance using reader-generated free-text image descriptions and to assess the effect of reader expertise on this LLM-augmented diagnostic workflow. METHODS. This retrospective study included 93 cases (encompassing radiographic, CT, MRI, and PET/CT images) from the Korean Society of Thoracic Radiology quiz platform from January 2014 to December 2017. Five differential diagnoses (the correct diagnosis and four distracters) were assembled for each case. Ten readers (five thoracic radiologists and five radiology residents) independently interpreted cases. In session 1, readers selected the most likely diagnosis and provided a free-text description of key findings. An LLM (Gemini 3.0 Pro) received as input the free-text description-without case images-and generated output that ranked the five differential diagnoses along with explanatory rationales for the top-three options. In session 2, readers were provided the LLM output from their own free-text description and reselected a most likely diagnosis. LLM performance using images-without free-text descriptions-was also assessed. LLM accuracy was determined using top-ranked diagnoses. Reader groups were compared using generalized estimating equations. RESULTS. LLM accuracy was 52.7% when images were inputted and 63.9% when reader-generated descriptions were inputted. LLM accuracy was greater using descriptions generated by thoracic radiologists than by residents (67.3% vs. 60.4%; p < .001). From session 1 to session 2, accuracy increased from 56.3% to 65.6% for thoracic radiologists and from 42.4% to 58.5% for residents, respectively. Improvement in accuracy between sessions was greater for residents than thoracic radiologists (16.1 vs 9.2 percentage points; p = .02). Residents, compared with thoracic radiologists, had a greater rate of accepting LLM-favored diagnoses (73.2% vs 48.9%; p < .001), including a greater rate of switching to an incorrect diagnosis after a misleading LLM output (60.6% vs 32.4%; p = .009). CONCLUSION. The text-based LLM-assisted workflow yielded improved reader accuracy, although this was heavily influenced by reader expertise. CLINICAL IMPACT. The utility of human-in-the-loop workflows arises from dynamic reader-LLM interactions shaped by the expertise of the operator formulating model inputs and critically evaluating model outputs.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, 101 Daehakro, Jongno-gu, Seoul, 03080, Korea.
- Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea.
- Department of Radiology, Uijeongbu Eulji University Hospital and Daejeon Eulji University, Daejeon, Korea.
- Department of Radiology, National Cancer Center, Goyang, Korea.
- Department of Radiology, National Jewish Health, Denver, CO.
Zitieren
APA
Song, J., Ko, H., Han, D. H., Hwang, E. J., Cho, H. S., Lee, J. Y., Jeong, W. G., Yoon, S. H., Kim, H., Kim, J., Park, J., Park, J., Son, Y., Lee, J. E., & Lee, T. (2026). Human-in-the-Loop Large Language Model–Augmented Diagnostic Reasoning in Thoracic Imaging: Impact of Radiologic Expertise. American Journal of Roentgenology. https://doi.org/10.2214/ajr.26.34999
Vancouver
Song J, Ko H, Han DH, Hwang EJ, Cho HS, Lee JY, et al. Human-in-the-Loop Large Language Model–Augmented Diagnostic Reasoning in Thoracic Imaging: Impact of Radiologic Expertise. American Journal of Roentgenology. 2026. doi:10.2214/ajr.26.34999
BibTeX
@article{Song2026,
author = {Song, Jiyoung and Ko, Hongseok and Han, Dae Hee and Hwang, Eui Jin and Cho, Hye Soo and Lee, Ji Young and Jeong, Won Gi and Yoon, Soon Ho and Kim, Hyungjin and Kim, Jiho and Park, Jongwoo and Park, Jiyoon and Son, Youbin and Lee, Joo Eon and Lee, Taehee},
title = {Human-in-the-Loop Large Language Model–Augmented Diagnostic Reasoning in Thoracic Imaging: Impact of Radiologic Expertise},
journal = {American Journal of Roentgenology},
year = {2026},
doi = {10.2214/ajr.26.34999}
}
EUVIMED