EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

ICEv2: Interpretability, Comprehensiveness, and Explainability in Vision Transformer

Hoyoung Choi, Seungwan Jin, Kyungsik Han

International Journal of Computer Vision · 2025-05 · 133 (5) · 2487-2504

DOI: 10.1007/s11263-024-02290-6 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 26.11.2024. Verfasst von Hoyoung Choi und 2 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Information & communications Technology Planning & Evaluation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision transformers use [CLS] token to predict image classes. Their explainability visualization has been studied using relevant information from the [CLS] token or focusing on attention scores during self-attention. However, such visualization is challenging because of the dependence of the interpretability of a vision transformer on skip connections and attention operators, the instability of non-linearities in the learning process, and the limited reflection of self-attention scores on relevance. We argue that the output patch embeddings in a vision transformer preserve the image information of each patch location, which can facilitate the prediction of an image class. In this paper, we propose ICEv2 (ICEv2: I̲\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\underline{\varvec{I}}}}}$$\end{document}nterpretability, C̲\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\underline{\varvec{C}}}}}$$\end{document}omprehensiveness, and E̲\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\underline{\varvec{E}}}}}$$\end{document}xplainability in Vision Transformer), an explainability visualization method that addresses the limitations of ICE (i.e., high dependence of hyperparameters on performance and the inability to preserve the model’s properties) by minimizing the number of training encoder layers, redesigning the MLP layer, and optimizing hyperparameters along with various model size. Overall, ICEv2 shows higher efficiency, performance, robustness, and scalability than ICE. On the ImageNet-Segmentation dataset, ICEv2 outperformed all explainability visualization methods in all cases depending on the model size. On the Pascal VOC dataset, ICEv2 outperformed both self-supervised and supervised methods on Jaccard similarity. In the unsupervised single object discovery, where untrained classes are present in the images, ICEv2 effectively distinguished between foreground and background, showing performance comparable to the previous state-of-the-art. Lastly, ICEv2 can be trained with significantly lower training computational complexity.

Förderung

  • Information & communications Technology Planning & Evaluation

Zitieren

APA

Choi, H., Jin, S., & Han, K. (2025). ICEv2: Interpretability, Comprehensiveness, and Explainability in Vision Transformer. International Journal of Computer Vision, 133(5), 2487-2504. https://doi.org/10.1007/s11263-024-02290-6

Vancouver

Choi H, Jin S, Han K. ICEv2: Interpretability, Comprehensiveness, and Explainability in Vision Transformer. International Journal of Computer Vision. 2025;133(5):2487-2504. doi:10.1007/s11263-024-02290-6

BibTeX
@article{Choi2025,
  author = {Choi, Hoyoung and Jin, Seungwan and Han, Kyungsik},
  title = {ICEv2: Interpretability, Comprehensiveness, and Explainability in Vision Transformer},
  journal = {International Journal of Computer Vision},
  year = {2025},
  volume = {133},
  number = {5},
  pages = {2487--2504},
  doi = {10.1007/s11263-024-02290-6}
}
Kostenlos starten →