Revisiting Change Captioning from Self-supervised Global-Part Alignment
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (6) · 5892-5900
DOI: 10.1609/aaai.v39i6.32629 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Feixiao Lv und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).
Abstract (Originalsprache)
The goal of image change captioning is to capture the content differences between two images and describe them in natural language. The key is how to learn stable content changes from noise such as viewpoint and image structure. However, current work mostly focuses on identifying changes, and the influence of global noise leads to unstable recognition of global features. In order to tackle this problem, we propose a Self-supervised Global-Part Alignment (SSGPA) network and revisit the image change captioning task by enhancing the construction process of overall image global features, enabling the model to integrate global changes such as viewpoint into local changes, and to detect and describe changes in the image through alignment. Concretely, we first design a Global-Part Transport Alignment mechanism to enhance global features and learn stable content changes through a self-supervised method of optimal transport. Further, we design a Change Fusion Adapter with pre-trained vision-language model to enhance the similar parts features of paired images, thereby enhancing global features, and expanding content changes. Extensive experiments show our method achieves the state-of-the-art results on four datasets.
Zitieren
APA
Lv, F., Wang, R., & Jing, L. (2025). Revisiting Change Captioning from Self-supervised Global-Part Alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 39(6), 5892-5900. https://doi.org/10.1609/aaai.v39i6.32629
Vancouver
Lv F, Wang R, Jing L. Revisiting Change Captioning from Self-supervised Global-Part Alignment. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(6):5892-5900. doi:10.1609/aaai.v39i6.32629
BibTeX
@article{Lv2025,
author = {Lv, Feixiao and Wang, Rui and Jing, Lihua},
title = {Revisiting Change Captioning from Self-supervised Global-Part Alignment},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {6},
pages = {5892--5900},
doi = {10.1609/aaai.v39i6.32629}
}
EUVIMED