EUVIMEDEUVIMED entdecken
Crossref

Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints

Ming Dai, Jian Li, Jiedong Zhuang, Xian Zhang, Wankou Yang

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (3) · 2618-2626

DOI: 10.1609/aaai.v39i3.32265 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Ming Dai und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-task visual grounding involves the simultaneous execution of localization and segmentation in images based on textual expressions. The majority of advanced methods predominantly focus on transformer-based multimodal fusion, aiming to extract robust multimodal representations. However, ambiguity between referring expression comprehension (REC) and referring image segmentation (RIS) is error-prone, leading to inconsistencies between multi-task predictions. Besides, insufficient multimodal understanding directly contributes to biased target perception. To overcome these challenges, we propose a Coarse-to-fine Consistency Constraints Visual Grounding architecture (C3VG), which integrates implicit and explicit modeling approaches within a two-stage framework. Initially, query and pixel decoders are employed to generate preliminary detection and segmentation outputs, a process referred to as the Rough Semantic Perception (RSP) stage. These coarse predictions are subsequently refined through the proposed Mask-guided Interaction Module (MIM) and a novel explicit bidirectional consistency constraint loss to ensure consistent representations across tasks, which we term the Refined Consistency Interaction (RCI) stage. Furthermore, to address the challenge of insufficient multimodal understanding, we leverage pre-trained models based on visual-linguistic fusion representations. Empirical evaluations on the RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate the efficacy and soundness of C3VG, which significantly outperforms state-of-the-art REC and RIS methods by a substantial margin.

Zitieren

APA

Dai, M., Li, J., Zhuang, J., Zhang, X., & Yang, W. (2025). Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints. Proceedings of the AAAI Conference on Artificial Intelligence, 39(3), 2618-2626. https://doi.org/10.1609/aaai.v39i3.32265

Vancouver

Dai M, Li J, Zhuang J, Zhang X, Yang W. Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(3):2618-2626. doi:10.1609/aaai.v39i3.32265

BibTeX
@article{Dai2025,
  author = {Dai, Ming and Li, Jian and Zhuang, Jiedong and Zhang, Xian and Yang, Wankou},
  title = {Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {3},
  pages = {2618--2626},
  doi = {10.1609/aaai.v39i3.32265}
}
Kostenlos starten →