EUVIMEDEUVIMED entdecken
Crossref

CMIRNet: Cross-Modal Interactive Reasoning Network for Referring Image Segmentation

Mingzhu Xu, Tianxiang Xiao, Yutong Liu, Haoyu Tang, Yupeng Hu, Liqiang Nie

IEEE Transactions on Circuits and Systems for Video Technology · 2025-04 · 35 (4) · 3234-3249

DOI: 10.1109/tcsvt.2024.3508752 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-04. Verfasst von Mingzhu Xu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 77 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Shandong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Referring Image Segmentation (RIS) aims to semantically segment the target object (referent) in alignment with the provided natural language query. Existing works still suffer from that the non-referent was segmented mistakenly, which can be attributed to the insufficient comprehension of vision and language. To tackle this problem, we propose a Cross-Modal Interactive Reasoning Network (CMIRNet) to explore semantic information that consistently existed between vision and language. Specifically, we first devise a novel Text-Guided Multi-Modality Joint Encoder (TGMM-JE), where the key expression can be extracted and the important visual features will be encoded under the continuous guidance of language expression. Then, we design a Cross-Graph Interactive Positioning (CGIP) module to locate the key pixels of the referent object in deepest layer. The multi-modality graph data is constructed between visual and linguistic features, and the important pixels can be positioned from cross-graph interaction and intra-graph reasoning. Finally, a novel Cross-Modal Attention Enhanced DEcoder (CMAE-DE) is dedicated to refine the referent object mask from coarse to fine progressively, where hybrid cross modal attentions are explored to enhance the representation of referent object. Extensive ablation studies validate the efficacy of our key modules and comprehensive experimental results show the superiority of our proposed model over 22 state-of-the-art (SOTA) models.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Shandong Province

Zitieren

APA

Xu, M., Xiao, T., Liu, Y., Tang, H., Hu, Y., & Nie, L. (2025). CMIRNet: Cross-Modal Interactive Reasoning Network for Referring Image Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 35(4), 3234-3249. https://doi.org/10.1109/tcsvt.2024.3508752

Vancouver

Xu M, Xiao T, Liu Y, Tang H, Hu Y, Nie L. CMIRNet: Cross-Modal Interactive Reasoning Network for Referring Image Segmentation. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(4):3234-3249. doi:10.1109/tcsvt.2024.3508752

BibTeX
@article{Xu2025,
  author = {Xu, Mingzhu and Xiao, Tianxiang and Liu, Yutong and Tang, Haoyu and Hu, Yupeng and Nie, Liqiang},
  title = {CMIRNet: Cross-Modal Interactive Reasoning Network for Referring Image Segmentation},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {4},
  pages = {3234--3249},
  doi = {10.1109/tcsvt.2024.3508752}
}
Kostenlos starten →