Evaluating Generative Artificial Intelligence as an Educational Tool for Radiology Resident Report Drafting
Journal of the American College of Radiology · 2026-05 · 23 (5) · 818-826
DOI: 10.1016/j.jacr.2025.12.024 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American College of Radiology, 2026-05. Verfasst von Antonio Verdone und 12 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, New York University Grossman School of Medicine, New York, New York. Hauptthemen laut MEDLINE: Radiology, Internship and Residency, Artificial Intelligence. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: NIBIB NIH HHS, NIMH NIH HHS, National Institute of Biomedical Imaging and Bioengineering.
Abstract (Originalsprache)
OBJECTIVE: Radiology residents require timely, personalized feedback to develop accurate image analysis and reporting skills. Increasing clinical workload often limits attendings' ability to provide guidance. This study evaluates a HIPAA-compliant Generative Pretrained Transformer (GPT)-4o system that delivers automated feedback on breast imaging reports drafted by residents in real clinical settings. METHODS: We analyzed 5,000 resident-attending report pairs from routine practice at a multisite US health system. GPT-4o was prompted with clinical instructions to identify common errors and provide feedback. A reader study using 100 report pairs was conducted. Four attending radiologists and four residents independently reviewed each pair, determined whether predefined error types were present, and rated GPT-4o's feedback as helpful or not. Agreement between GPT and readers was assessed using percent match. Interreader reliability was measured with Krippendorff's α. Educational value was measured as the proportion of cases rated helpful. RESULTS: Three common error types were identified: (1) omission or addition of key findings, (2) incorrect use or omission of technical descriptors, and (3) final assessment inconsistent with findings. GPT-4o showed strong agreement with attending consensus: 90.5%, 78.3%, and 90.4% (Cohen's κ: 0.790, 0.550, and 0.615) across error types. Interreader reliability among all eight readers showed moderate to substantial variability (α = 0.767, 0.595, 0.567). When each reader was individually replaced with GPT-4o and interreader agreement among seven readers and GPT was recalculated, the effect was not statistically significant (Δ = -0.004 to 0.002, all P > .05). GPT's feedback was rated helpful in most cases: 89.8%, 83.0%, and 92.0%. DISCUSSION: ChatGPT-4o can reliably identify key educational errors. It may serve as a scalable tool to support radiology education.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology, New York University Grossman School of Medicine, New York, New York.
- New York University Grossman School of Medicine, New York, New York.
- Department of Radiology, New York University Grossman School of Medicine, New York, New York; New York University, Center for Data Science, New York, New York. Electronic address: Yiqiu.Shen@nyulangone.org.
Förderung
- NIBIB NIH HHS
- NIMH NIH HHS
- National Institute of Biomedical Imaging and Bioengineering
- National Institutes of Health
- NYU Grossman School of Medicine
- New York University Grossman School of Medicine
Zitieren
APA
Verdone, A., Cardall, A., Siddiqui, F., Nashawaty, M., Rigau, D., Kwon, Y., Yousef, M., Patel, S., Kieturakis, A., Kim, E., Heacock, L., Reig, B., & Shen, Y. (2026). Evaluating Generative Artificial Intelligence as an Educational Tool for Radiology Resident Report Drafting. Journal of the American College of Radiology, 23(5), 818-826. https://doi.org/10.1016/j.jacr.2025.12.024
Vancouver
Verdone A, Cardall A, Siddiqui F, Nashawaty M, Rigau D, Kwon Y, et al. Evaluating Generative Artificial Intelligence as an Educational Tool for Radiology Resident Report Drafting. Journal of the American College of Radiology. 2026;23(5):818-826. doi:10.1016/j.jacr.2025.12.024
BibTeX
@article{Verdone2026,
author = {Verdone, Antonio and Cardall, Aidan and Siddiqui, Fardeen and Nashawaty, Motaz and Rigau, Danielle and Kwon, Youngjoon and Yousef, Mira and Patel, Shalin and Kieturakis, Alex and Kim, Eric and Heacock, Laura and Reig, Beatriu and Shen, Yiqiu},
title = {Evaluating Generative Artificial Intelligence as an Educational Tool for Radiology Resident Report Drafting},
journal = {Journal of the American College of Radiology},
year = {2026},
volume = {23},
number = {5},
pages = {818--826},
doi = {10.1016/j.jacr.2025.12.024}
}
EUVIMED