Make Graph-Based Referring Expression Comprehension Great Again Through Expression-Guided Dynamic Gating and Regression
IEEE Transactions on Multimedia · 2025 · 27 · 1950-1961
DOI: 10.1109/tmm.2024.3521844 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Multimedia, 2025. Verfasst von Jingcheng Ke und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 54 Literaturangaben. Gefördert von: National Science and Technology Council, Qualcomm, Taiwan University Research Collaboration Project.
Abstract (Originalsprache)
One common belief is that with complex models and pre-training on large-scale datasets, transformer-based methods for referring expression comprehension (REC) perform much better than existing graph-based methods. We observe that since most graph-based methods adopt an off-the-shelf detector to locate candidate objects (i.e., regions detected by the object detector), they face two challenges that result in subpar performance: (1) the presence of significant noise caused by numerous irrelevant objects during reasoning, and (2) inaccurate localization outcomes attributed to the provided detector. To address these issues, we introduce a plug-and-adapt module guided by sub-expressions, called dynamic gate constraint (DGC), which can adaptively disable irrelevant proposals and their connections in graphs during reasoning. We further introduce an expression-guided regression strategy (EGR) to refine location prediction. Extensive experimental results on the RefCOCO, RefCOCO+, RefCOCOg, Flickr30 K, RefClef, and Ref-reasoning datasets demonstrate the effectiveness of the DGC module and the EGR strategy in consistently boosting the performances of various graph-based REC methods. Without any pretaining, the proposed graph-based method achieves better performance than the state-of-the-art (SOTA) transformer-based methods.
Förderung
- National Science and Technology Council
- Qualcomm
- Taiwan University Research Collaboration Project
Zitieren
APA
Ke, J., Wang, D., Chen, J., Jhuo, I., Lin, C., & Lin, Y. (2025). Make Graph-Based Referring Expression Comprehension Great Again Through Expression-Guided Dynamic Gating and Regression. IEEE Transactions on Multimedia, 27, 1950-1961. https://doi.org/10.1109/tmm.2024.3521844
Vancouver
Ke J, Wang D, Chen J, Jhuo I, Lin C, Lin Y. Make Graph-Based Referring Expression Comprehension Great Again Through Expression-Guided Dynamic Gating and Regression. IEEE Transactions on Multimedia. 2025;27:1950-1961. doi:10.1109/tmm.2024.3521844
BibTeX
@article{Ke2025,
author = {Ke, Jingcheng and Wang, Dele and Chen, Jun-Cheng and Jhuo, I-Hong and Lin, Chia-Wen and Lin, Yen-Yu},
title = {Make Graph-Based Referring Expression Comprehension Great Again Through Expression-Guided Dynamic Gating and Regression},
journal = {IEEE Transactions on Multimedia},
year = {2025},
volume = {27},
pages = {1950--1961},
doi = {10.1109/tmm.2024.3521844}
}
EUVIMED