Evaluating progress in automatic chest X-ray radiology report generation
Patterns · 2023-09 · 4 (9) · 100802
DOI: 10.1016/j.patter.2023.100802 ↗Kurz gefasst
Fachartikel in der Zeitschrift Patterns, 2023-09. Verfasst von Feiyang Yu und 12 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Stanford University, Stanford, CA 94305, USA. Die Arbeit wurde bisher 179-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: NHLBI NIH HHS, National Institutes of Health, National Institute of Biomedical Imaging and Bioengineering.
Abstract (Originalsprache)
Artificial intelligence (AI) models for automatic generation of narrative radiology reports from images have the potential to enhance efficiency and reduce the workload of radiologists. However, evaluating the correctness of these reports requires metrics that can capture clinically pertinent differences. In this study, we investigate the alignment between automated metrics and radiologists' scoring of errors in report generation. We address the limitations of existing metrics by proposing new metrics, RadGraph F1 and RadCliQ, which demonstrate stronger correlation with radiologists' evaluations. In addition, we analyze the failure modes of the metrics to understand their limitations and provide guidance for metric selection and interpretation. This study establishes RadGraph F1 and RadCliQ as meaningful metrics for guiding future research in radiology report generation.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science, Stanford University, Stanford, CA 94305, USA.
- Department of Radiology, Brigham and Women's Hospital, Boston, MA 02115, USA.
- Department of Radiology, Boston Children's Hospital, Harvard Medical School, Boston, MA 02115, USA.
- Cardiothoracic Radiology Group, Hospital Israelita Albert Einstein, São Paulo, São Paulo 05652, Brazil.
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON M5T 3M7, Canada.
- AIMI Center, Stanford University, Stanford, CA 94304, USA.
Förderung
- NHLBI NIH HHS
- National Institutes of Health
- National Institute of Biomedical Imaging and Bioengineering
- Gordon and Betty Moore Foundation
- International Business Machines Corporation
Zitieren
APA
Yu, F., Endo, M., Krishnan, R., Pan, I., Tsai, A., Reis, E. P., Fonseca, E. K. U. N., Lee, H. M. H., Abad, Z. S. H., Ng, A. Y., Langlotz, C. P., Venugopal, V. K., & Rajpurkar, P. (2023). Evaluating progress in automatic chest X-ray radiology report generation. Patterns, 4(9), 100802. https://doi.org/10.1016/j.patter.2023.100802
Vancouver
Yu F, Endo M, Krishnan R, Pan I, Tsai A, Reis EP, et al. Evaluating progress in automatic chest X-ray radiology report generation. Patterns. 2023;4(9):100802. doi:10.1016/j.patter.2023.100802
BibTeX
@article{Yu2023,
author = {Yu, Feiyang and Endo, Mark and Krishnan, Rayan and Pan, Ian and Tsai, Andy and Reis, Eduardo Pontes and Fonseca, Eduardo Kaiser Ururahy Nunes and Lee, Henrique Min Ho and Abad, Zahra Shakeri Hossein and Ng, Andrew Y and Langlotz, Curtis P and Venugopal, Vasantha Kumar and Rajpurkar, Pranav},
title = {Evaluating progress in automatic chest X-ray radiology report generation},
journal = {Patterns},
year = {2023},
volume = {4},
number = {9},
pages = {100802},
doi = {10.1016/j.patter.2023.100802}
}
EUVIMED