EUVIMEDEUVIMED entdecken
Crossref

IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis

Yuji Wang, Jingchen Ni, Yong Liu, Chun Yuan, Yansong Tang

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (8) · 8159-8168

DOI: 10.1609/aaai.v39i8.32880 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Yuji Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Zero-shot Referring Image Segmentation (RIS) identifies the instance mask that best aligns with a specified referring expression without training and fine-tuning, significantly reducing the labor-intensive annotation process. Despite achieving commendable results, previous CLIP-based models have a critical drawback: the models exhibit a notable reduction in their capacity to discern relative spatial relationships of objects. This is because they generate all possible masks on an image and evaluate each masked region for similarity to the given expression, often resulting in decreased sensitivity to direct positional clues in text inputs. Moreover, most methods have weak abilities to manage relationships between primary words and their contexts, causing confusion and reduced accuracy in identifying the correct target region. To address these challenges, we propose IteRPrimE (Iterative Grad-CAM Refinement and Primary word Emphasis), which leverages a saliency heatmap through Grad-CAM from a Vision-Language Pre-trained (VLP) model for image-text matching. An iterative Grad-CAM refinement strategy is introduced to progressively enhance the model's focus on the target region and overcome positional insensitivity, creating a self-correcting effect. Additionally, we design the Primary Word Emphasis module to help the model handle complex semantic relations, enhancing its ability to attend to the intended object. Extensive experiments conducted on the RefCOCO/+/g, and PhraseCut benchmarks demonstrate that IteRPrimE outperforms previous SOTA zero-shot methods, particularly excelling in out-of-domain scenarios.

Zitieren

APA

Wang, Y., Ni, J., Liu, Y., Yuan, C., & Tang, Y. (2025). IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis. Proceedings of the AAAI Conference on Artificial Intelligence, 39(8), 8159-8168. https://doi.org/10.1609/aaai.v39i8.32880

Vancouver

Wang Y, Ni J, Liu Y, Yuan C, Tang Y. IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(8):8159-8168. doi:10.1609/aaai.v39i8.32880

BibTeX
@article{Wang2025,
  author = {Wang, Yuji and Ni, Jingchen and Liu, Yong and Yuan, Chun and Tang, Yansong},
  title = {IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasis},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {8},
  pages = {8159--8168},
  doi = {10.1609/aaai.v39i8.32880}
}
Kostenlos starten →