Gradient-Based Instance-Specific Visual Explanations for Object Specification and Object Discrimination
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-09 · 46 (9) · 5967-5985
DOI: 10.1109/tpami.2024.3380604 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-09. Verfasst von Chenyang Zhao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 90 Literaturangaben. Gefördert von: Council Hong Kong, City University of Hong Kong.
Abstract (Originalsprache)
We propose the gradient-weighted Object Detector Activation Maps (ODAM), a visual explanation technique for interpreting the predictions of object detectors. Utilizing the gradients of detector targets flowing into the intermediate feature maps, ODAM produces heat maps that show the influence of regions on the detector's decision for each predicted attribute. Compared to previous works on classification activation maps (CAM), ODAM generates instance-specific explanations rather than class-specific ones. We show that ODAM is applicable to one-stage, two-stage, and transformer-based detectors with different types of detector backbones and heads, and produces higher-quality visual explanations than the state-of-the-art in terms of both effectiveness and efficiency. We discuss two explanation tasks for object detection: 1) object specification: what is the important region for the prediction? 2) object discrimination: which object is detected? Aiming at these two aspects, we present a detailed analysis of the visual explanations of detectors and carry out extensive experiments to validate the effectiveness of the proposed ODAM. Furthermore, we investigate user trust on the explanation maps, how well the visual explanations of object detectors agrees with human explanations, as measured through human eye gaze, and whether this agreement is related with user trust. Finally, we also propose two applications, ODAM-KD and ODAM-NMS, based on these two abilities of ODAM. ODAM-KD utilizes the object specification of ODAM to generate top-down attention for key predictions and instruct the knowledge distillation of object detection. ODAM-NMS considers the location of the model's explanation for each prediction to distinguish the duplicate detected objects. A training scheme, ODAM-Train, is proposed to improve the quality on object discrimination, and help with ODAM-NMS.
Förderung
- Council Hong Kong
- City University of Hong Kong
Zitieren
APA
Zhao, C., Hsiao, J. H., & Chan, A. B. (2024). Gradient-Based Instance-Specific Visual Explanations for Object Specification and Object Discrimination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9), 5967-5985. https://doi.org/10.1109/tpami.2024.3380604
Vancouver
Zhao C, Hsiao JH, Chan AB. Gradient-Based Instance-Specific Visual Explanations for Object Specification and Object Discrimination. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(9):5967-5985. doi:10.1109/tpami.2024.3380604
BibTeX
@article{Zhao2024,
author = {Zhao, Chenyang and Hsiao, Janet H and Chan, Antoni B},
title = {Gradient-Based Instance-Specific Visual Explanations for Object Specification and Object Discrimination},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {9},
pages = {5967--5985},
doi = {10.1109/tpami.2024.3380604}
}
EUVIMED