EUVIMEDEUVIMED entdecken
Crossref

Look Around Before Locating: Considering Content and Structure Information for Visual Grounding

Shiyi Zheng, Peizhi Zhao, Zhilong Zheng, Peihang He, Haonan Cheng, Yi Cai, Qingbao Huang

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (2) · 1656-1664

DOI: 10.1609/aaai.v39i2.32158 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Shiyi Zheng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As a long-term challenge and fundamental requirement in vision and language tasks, visual grounding aims to localize a target referred by a natural language query. The regional annotations form a superficial correlation between the subject of expression and some common visual entities, which hinder models from comprehending the linguistic content and structure. However, current one-stage methods struggle to uniformly model the visual and linguistic structure due to the structural gap between continuous image patches and discrete text tokens. In this paper, we propose a semi-structured reasoning framework for visual grounding to gradually comprehend the linguistic content and structure. Specifically, we devise a cross-modal content alignment module to effectively align unlabeled contextual information into a stable semantic space corrected by token-level prior knowledge obtained with CLIP. A multi-branch modulated localization module is also established to obtain modulation grounding by linguistic structure. Through a soft split mechanism, our method can destructure the expression into a fixed semi-structure (i.e., subject and context) while ensuring the completeness of linguistic content. Our method is thus capable of building a semi-structured reasoning system to effectively comprehend the linguistic content and structure by content alignment and structure modulated grounding. Experimental results on five widely-used datasets validate the performance improvements of our proposed method.

Zitieren

APA

Zheng, S., Zhao, P., Zheng, Z., He, P., Cheng, H., Cai, Y., & Huang, Q. (2025). Look Around Before Locating: Considering Content and Structure Information for Visual Grounding. Proceedings of the AAAI Conference on Artificial Intelligence, 39(2), 1656-1664. https://doi.org/10.1609/aaai.v39i2.32158

Vancouver

Zheng S, Zhao P, Zheng Z, He P, Cheng H, Cai Y, et al. Look Around Before Locating: Considering Content and Structure Information for Visual Grounding. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(2):1656-1664. doi:10.1609/aaai.v39i2.32158

BibTeX
@article{Zheng2025,
  author = {Zheng, Shiyi and Zhao, Peizhi and Zheng, Zhilong and He, Peihang and Cheng, Haonan and Cai, Yi and Huang, Qingbao},
  title = {Look Around Before Locating: Considering Content and Structure Information for Visual Grounding},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {2},
  pages = {1656--1664},
  doi = {10.1609/aaai.v39i2.32158}
}
Kostenlos starten →