EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models

Yao Zhu, Yunjian Zhang, Zizhe Wang, Xiu Yan, Peng Sun, Xiangyang Ji

Transactions of the Association for Computational Linguistics · 18.07.2025 · 13 · 744-759

DOI: 10.1162/tacl_a_00756 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 18.07.2025. Verfasst von Yao Zhu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Amidst the rapid advancement of artificial intelligence, research on large vision-language models (LVLMs) has emerged as a pivotal area. However, understanding their internal mechanisms remains challenging due to the limitations of existing interpretability methods, especially regarding faithfulness and plausibility. To address this, we first construct a human response interpretability dataset that evaluates the plausibility of model explanations by comparing the attention regions between the model and humans when answering the same questions. We then propose a patchwise cooperative game-based interpretability method for LVLMs, which employs Shapley values to quantify the impact of individual image patches on generation likelihood and enhances computational efficiency through a single input approximation approach. Experimental results demonstrate our method’s faithfulness, plausibility, and robustness. Our method provides researchers with deeper insights into model behavior, allowing for an examination of the specific image regions each layer relies on during response generation, ultimately enhancing model reliability. Our code is available at https://github.com/ZY123-GOOD/Patchwise_Cooperative.

Zitieren

APA

Zhu, Y., Zhang, Y., Wang, Z., Yan, X., Sun, P., & Ji, X. (2025). Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models. Transactions of the Association for Computational Linguistics, 13, 744-759. https://doi.org/10.1162/tacl_a_00756

Vancouver

Zhu Y, Zhang Y, Wang Z, Yan X, Sun P, Ji X. Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models. Transactions of the Association for Computational Linguistics. 2025;13:744-759. doi:10.1162/tacl_a_00756

BibTeX
@article{Zhu2025,
  author = {Zhu, Yao and Zhang, Yunjian and Wang, Zizhe and Yan, Xiu and Sun, Peng and Ji, Xiangyang},
  title = {Patchwise Cooperative Game-based Interpretability Method for Large Vision-language Models},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2025},
  volume = {13},
  pages = {744--759},
  doi = {10.1162/tacl_a_00756}
}
Kostenlos starten →