Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses
Journal of Imaging Informatics in Medicine · 06.05.2024 · 37 (5) · 2490-2499
DOI: 10.1007/s10278-024-01136-4 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Imaging Informatics in Medicine, online erschienen am 06.05.2024. Verfasst von Kesavan Venkatesh und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA. Hauptthemen laut MEDLINE: Musculoskeletal Diseases. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Division of Computing and Communication Foundations, Whiting School of Engineering, Johns Hopkins University.
Abstract (Originalsprache)
Saliency maps are popularly used to "explain" decisions made by modern machine learning models, including deep convolutional neural networks (DCNNs). While the resulting heatmaps purportedly indicate important image features, their "trustworthiness," i.e., utility and robustness, has not been evaluated for musculoskeletal imaging. The purpose of this study was to systematically evaluate the trustworthiness of saliency maps used in disease diagnosis on upper extremity X-ray images. The underlying DCNNs were trained using the Stanford MURA dataset. We studied four trustworthiness criteria-(1) localization accuracy of abnormalities, (2) repeatability, (3) reproducibility, and (4) sensitivity to underlying DCNN weights-across six different gradient-based saliency methods (Grad-CAM (GCAM), gradient explanation (GRAD), integrated gradients (IG), Smoothgrad (SG), smooth IG (SIG), and XRAI). Ground-truth was defined by the consensus of three fellowship-trained musculoskeletal radiologists who each placed bounding boxes around abnormalities on a holdout saliency test set. Compared to radiologists, all saliency methods showed inferior localization (AUPRCs: 0.438 (SG)-0.590 (XRAI); average radiologist AUPRC: 0.816), repeatability (IoUs: 0.427 (SG)-0.551 (IG); average radiologist IOU: 0.613), and reproducibility (IoUs: 0.250 (SG)-0.502 (XRAI); average radiologist IOU: 0.613) on abnormalities such as fractures, orthopedic hardware insertions, and arthritis. Five methods (GCAM, GRAD, IG, SG, XRAI) passed the sensitivity test. Ultimately, no saliency method met all four trustworthiness criteria; therefore, we recommend caution and rigorous evaluation of saliency maps prior to their clinical use.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA.
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD, USA.
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD, USA. pyi@som.umaryl…
ORCID-Profile: Kesavan Venkatesh, Jeremias Sulam, Paul H Yi
Förderung
- Division of Computing and Communication Foundations
- Whiting School of Engineering, Johns Hopkins University
Zitieren
APA
Venkatesh, K., Mutasa, S., Moore, F., Sulam, J., & Yi, P. H. (2024). Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses. Journal of Imaging Informatics in Medicine, 37(5), 2490-2499. https://doi.org/10.1007/s10278-024-01136-4
Vancouver
Venkatesh K, Mutasa S, Moore F, Sulam J, Yi PH. Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses. Journal of Imaging Informatics in Medicine. 2024;37(5):2490-2499. doi:10.1007/s10278-024-01136-4
BibTeX
@article{Venkatesh2024,
author = {Venkatesh, Kesavan and Mutasa, Simukayi and Moore, Fletcher and Sulam, Jeremias and Yi, Paul H},
title = {Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses},
journal = {Journal of Imaging Informatics in Medicine},
year = {2024},
volume = {37},
number = {5},
pages = {2490--2499},
doi = {10.1007/s10278-024-01136-4}
}
EUVIMED