Look Around Before Locating: Considering Content and Structure Information for Visual Grounding
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (2) · 1656-1664
DOI: 10.1609/aaai.v39i2.32158 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Shiyi Zheng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref).
Abstract (Originalsprache)
As a long-term challenge and fundamental requirement in vision and language tasks, visual grounding aims to localize a target referred by a natural language query. The regional annotations form a superficial correlation between the subject of expression and some common visual entities, which hinder models from comprehending the linguistic content and structure. However, current one-stage methods struggle to uniformly model the visual and linguistic structure due to the structural gap between continuous image patches and discrete text tokens. In this paper, we propose a semi-structured reasoning framework for visual grounding to gradually comprehend the linguistic content and structure. Specifically, we devise a cross-modal content alignment module to effectively align unlabeled contextual information into a stable semantic space corrected by token-level prior knowledge obtained with CLIP. A multi-branch modulated localization module is also established to obtain modulation grounding by linguistic structure. Through a soft split mechanism, our method can destructure the expression into a fixed semi-structure (i.e., subject and context) while ensuring the completeness of linguistic content. Our method is thus capable of building a semi-structured reasoning system to effectively comprehend the linguistic content and structure by content alignment and structure modulated grounding. Experimental results on five widely-used datasets validate the performance improvements of our proposed method.
Zitieren
APA
Zheng, S., Zhao, P., Zheng, Z., He, P., Cheng, H., Cai, Y., & Huang, Q. (2025). Look Around Before Locating: Considering Content and Structure Information for Visual Grounding. Proceedings of the AAAI Conference on Artificial Intelligence, 39(2), 1656-1664. https://doi.org/10.1609/aaai.v39i2.32158
Vancouver
Zheng S, Zhao P, Zheng Z, He P, Cheng H, Cai Y, et al. Look Around Before Locating: Considering Content and Structure Information for Visual Grounding. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(2):1656-1664. doi:10.1609/aaai.v39i2.32158
BibTeX
@article{Zheng2025,
author = {Zheng, Shiyi and Zhao, Peizhi and Zheng, Zhilong and He, Peihang and Cheng, Haonan and Cai, Yi and Huang, Qingbao},
title = {Look Around Before Locating: Considering Content and Structure Information for Visual Grounding},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {2},
pages = {1656--1664},
doi = {10.1609/aaai.v39i2.32158}
}
EUVIMED