EUVIMEDEUVIMED entdecken
Open Access

Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses

Kesavan Venkatesh iD, Simukayi Mutasa, Fletcher Moore, Jeremias Sulam iD, Paul H Yi iD

Journal of Imaging Informatics in Medicine · 06.05.2024 · 37 (5) · 2490-2499

DOI: 10.1007/s10278-024-01136-4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Imaging Informatics in Medicine, online erschienen am 06.05.2024. Verfasst von Kesavan Venkatesh und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA. Hauptthemen laut MEDLINE: Musculoskeletal Diseases. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Division of Computing and Communication Foundations, Whiting School of Engineering, Johns Hopkins University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Saliency maps are popularly used to "explain" decisions made by modern machine learning models, including deep convolutional neural networks (DCNNs). While the resulting heatmaps purportedly indicate important image features, their "trustworthiness," i.e., utility and robustness, has not been evaluated for musculoskeletal imaging. The purpose of this study was to systematically evaluate the trustworthiness of saliency maps used in disease diagnosis on upper extremity X-ray images. The underlying DCNNs were trained using the Stanford MURA dataset. We studied four trustworthiness criteria-(1) localization accuracy of abnormalities, (2) repeatability, (3) reproducibility, and (4) sensitivity to underlying DCNN weights-across six different gradient-based saliency methods (Grad-CAM (GCAM), gradient explanation (GRAD), integrated gradients (IG), Smoothgrad (SG), smooth IG (SIG), and XRAI). Ground-truth was defined by the consensus of three fellowship-trained musculoskeletal radiologists who each placed bounding boxes around abnormalities on a holdout saliency test set. Compared to radiologists, all saliency methods showed inferior localization (AUPRCs: 0.438 (SG)-0.590 (XRAI); average radiologist AUPRC: 0.816), repeatability (IoUs: 0.427 (SG)-0.551 (IG); average radiologist IOU: 0.613), and reproducibility (IoUs: 0.250 (SG)-0.502 (XRAI); average radiologist IOU: 0.613) on abnormalities such as fractures, orthopedic hardware insertions, and arthritis. Five methods (GCAM, GRAD, IG, SG, XRAI) passed the sensitivity test. Ultimately, no saliency method met all four trustworthiness criteria; therefore, we recommend caution and rigorous evaluation of saliency maps prior to their clinical use.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Musculoskeletal DiseasesHumansNeural Networks, ComputerReproducibility of ResultsDeep LearningMachine LearningMusculoskeletal SystemArtificial Intelligence

Stichwörter der Autor:innen

Deep learningMusculoskeletal radiologySaliency methodsTrustworthy AI

Autor:innen und Institutionen

  • Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA.
  • University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD, USA.
  • University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD, USA. pyi@som.umaryl…

ORCID-Profile: Kesavan Venkatesh, Jeremias Sulam, Paul H Yi

Förderung

  • Division of Computing and Communication Foundations
  • Whiting School of Engineering, Johns Hopkins University

Zitieren

APA

Venkatesh, K., Mutasa, S., Moore, F., Sulam, J., & Yi, P. H. (2024). Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses. Journal of Imaging Informatics in Medicine, 37(5), 2490-2499. https://doi.org/10.1007/s10278-024-01136-4

Vancouver

Venkatesh K, Mutasa S, Moore F, Sulam J, Yi PH. Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses. Journal of Imaging Informatics in Medicine. 2024;37(5):2490-2499. doi:10.1007/s10278-024-01136-4

BibTeX
@article{Venkatesh2024,
  author = {Venkatesh, Kesavan and Mutasa, Simukayi and Moore, Fletcher and Sulam, Jeremias and Yi, Paul H},
  title = {Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses},
  journal = {Journal of Imaging Informatics in Medicine},
  year = {2024},
  volume = {37},
  number = {5},
  pages = {2490--2499},
  doi = {10.1007/s10278-024-01136-4}
}
Kostenlos starten →