Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models
Transactions of the Association for Computational Linguistics · 18.07.2025 · 13 · 744-759
DOI: 10.1162/tacl_a_00756 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 18.07.2025. Verfasst von Yao Zhu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Amidst the rapid advancement of artificial intelligence, research on large vision-language models (LVLMs) has emerged as a pivotal area. However, understanding their internal mechanisms remains challenging due to the limitations of existing interpretability methods, especially regarding faithfulness and plausibility. To address this, we first construct a human response interpretability dataset that evaluates the plausibility of model explanations by comparing the attention regions between the model and humans when answering the same questions. We then propose a patchwise cooperative game-based interpretability method for LVLMs, which employs Shapley values to quantify the impact of individual image patches on generation likelihood and enhances computational efficiency through a single input approximation approach. Experimental results demonstrate our method’s faithfulness, plausibility, and robustness. Our method provides researchers with deeper insights into model behavior, allowing for an examination of the specific image regions each layer relies on during response generation, ultimately enhancing model reliability. Our code is available at https://github.com/ZY123-GOOD/Patchwise_Cooperative.
Zitieren
APA
Zhu, Y., Zhang, Y., Wang, Z., Yan, X., Sun, P., & Ji, X. (2025). Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models. Transactions of the Association for Computational Linguistics, 13, 744-759. https://doi.org/10.1162/tacl_a_00756
Vancouver
Zhu Y, Zhang Y, Wang Z, Yan X, Sun P, Ji X. Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models. Transactions of the Association for Computational Linguistics. 2025;13:744-759. doi:10.1162/tacl_a_00756
BibTeX
@article{Zhu2025,
author = {Zhu, Yao and Zhang, Yunjian and Wang, Zizhe and Yan, Xiu and Sun, Peng and Ji, Xiangyang},
title = {Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models},
journal = {Transactions of the Association for Computational Linguistics},
year = {2025},
volume = {13},
pages = {744--759},
doi = {10.1162/tacl_a_00756}
}
EUVIMED