Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (3) · 2618-2626
DOI: 10.1609/aaai.v39i3.32265 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Ming Dai und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref).
Abstract (Originalsprache)
Multi-task visual grounding involves the simultaneous execution of localization and segmentation in images based on textual expressions. The majority of advanced methods predominantly focus on transformer-based multimodal fusion, aiming to extract robust multimodal representations. However, ambiguity between referring expression comprehension (REC) and referring image segmentation (RIS) is error-prone, leading to inconsistencies between multi-task predictions. Besides, insufficient multimodal understanding directly contributes to biased target perception. To overcome these challenges, we propose a Coarse-to-fine Consistency Constraints Visual Grounding architecture (C3VG), which integrates implicit and explicit modeling approaches within a two-stage framework. Initially, query and pixel decoders are employed to generate preliminary detection and segmentation outputs, a process referred to as the Rough Semantic Perception (RSP) stage. These coarse predictions are subsequently refined through the proposed Mask-guided Interaction Module (MIM) and a novel explicit bidirectional consistency constraint loss to ensure consistent representations across tasks, which we term the Refined Consistency Interaction (RCI) stage. Furthermore, to address the challenge of insufficient multimodal understanding, we leverage pre-trained models based on visual-linguistic fusion representations. Empirical evaluations on the RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate the efficacy and soundness of C3VG, which significantly outperforms state-of-the-art REC and RIS methods by a substantial margin.
Zitieren
APA
Dai, M., Li, J., Zhuang, J., Zhang, X., & Yang, W. (2025). Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints. Proceedings of the AAAI Conference on Artificial Intelligence, 39(3), 2618-2626. https://doi.org/10.1609/aaai.v39i3.32265
Vancouver
Dai M, Li J, Zhuang J, Zhang X, Yang W. Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(3):2618-2626. doi:10.1609/aaai.v39i3.32265
BibTeX
@article{Dai2025,
author = {Dai, Ming and Li, Jian and Zhuang, Jiedong and Zhang, Xian and Yang, Wankou},
title = {Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {3},
pages = {2618--2626},
doi = {10.1609/aaai.v39i3.32265}
}
EUVIMED