EUVIMEDEUVIMED entdecken
Crossref

Cross-Modal Recurrent Semantic Comprehension for Referring Image Segmentation

Chao Shang, Hongliang Li, Heqian Qiu, Qingbo Wu, Fanman Meng, Taijin Zhao, King Ngi Ngan

IEEE Transactions on Circuits and Systems for Video Technology · 2023-07 · 33 (7) · 3229-3242

DOI: 10.1109/tcsvt.2022.3231964 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-07. Verfasst von Chao Shang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Referring image segmentation aims to segment the target object from the image according to the description of language expression. Due to the diversity of language expressions, word sequences in different orders often express different semantic information. The previous methods focus more on matching different words to different visual regions in the image separately, ignoring the global semantic understanding of language expression based on the sequence structure. To address this problem, we redesign a new recurrent network structure for referring image segmentation, called Cross-Modal Recurrent Semantic Comprehension Network (CRSCNet), to obtain a more comprehensive global semantic understanding through iterative cross-modal semantic reasoning. Specifically, in each iteration, we first propose a Dynamic SepConv to extract relevant visual features guided by language and further propose Language Attentional Feature Modulation to improve the feature discriminability, then propose a Cross-Modal Semantic Reasoning module to perform global semantic reasoning by capturing both linguistic and visual information, and finally updates and corrects the visual features of the predicted object based on semantic information. Moreover, we further propose a Cross-Modal ASPP to capture richer visual information referred to in the global semantics of the language expression from larger receptive fields. Extensive experiments demonstrate that our proposed network significantly outperforms previous state-of-the-art methods on multiple datasets.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China

Zitieren

APA

Shang, C., Li, H., Qiu, H., Wu, Q., Meng, F., Zhao, T., & Ngan, K. N. (2023). Cross-Modal Recurrent Semantic Comprehension for Referring Image Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 33(7), 3229-3242. https://doi.org/10.1109/tcsvt.2022.3231964

Vancouver

Shang C, Li H, Qiu H, Wu Q, Meng F, Zhao T, et al. Cross-Modal Recurrent Semantic Comprehension for Referring Image Segmentation. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(7):3229-3242. doi:10.1109/tcsvt.2022.3231964

BibTeX
@article{Shang2023,
  author = {Shang, Chao and Li, Hongliang and Qiu, Heqian and Wu, Qingbo and Meng, Fanman and Zhao, Taijin and Ngan, King Ngi},
  title = {Cross-Modal Recurrent Semantic Comprehension for Referring Image Segmentation},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2023},
  volume = {33},
  number = {7},
  pages = {3229--3242},
  doi = {10.1109/tcsvt.2022.3231964}
}
Kostenlos starten →