EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MIPANet: optimizing RGB-D semantic segmentation through multi-modal interaction and pooling attention

Shuai Zhang, Minghong Xie

Frontiers in Physics · 31.05.2024 · 12

DOI: 10.3389/fphy.2024.1411559 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Physics, online erschienen am 31.05.2024. Verfasst von Shuai Zhang und 1 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The semantic segmentation of RGB-D images involves understanding objects appearances and spatial relationships within a scene, which necessitates careful consideration of multiple factors. In indoor scenes, the presence of diverse and disorderly objects, coupled with illumination variations and the influence of adjacent objects, can easily result in misclassifications of pixels, consequently affecting the outcome of semantic segmentation. We propose a Multi-modal Interaction and Pooling Attention Network (MIPANet) in response to these challenges. This network is designed to exploit the interactive synergy between RGB and depth modalities, aiming to enhance the utilization of complementary information and improve segmentation accuracy. Specifically, we incorporate a Multi-modal Interaction Module (MIM) into the deepest layers of the network. This module is engineered to facilitate the fusion of RGB and depth information, allowing for mutual enhancement and correction. Moreover, we introduce a Pooling Attention Module (PAM) at various stages of the encoder to enhance the features extracted by the network. The outputs of the PAMs at different stages are selectively integrated into the decoder through a refinement module to improve semantic segmentation performance. Experimental results demonstrate that MIPANet outperforms existing methods on two indoor scene datasets, NYU-Depth V2 and SUN-RGBD, by optimizing the insufficient information interaction between different modalities in RGB-D semantic segmentation. The source codes are available at https://github.com/2295104718/MIPANet .

Zitieren

APA

Zhang, S., & Xie, M. (2024). MIPANet: optimizing RGB-D semantic segmentation through multi-modal interaction and pooling attention. Frontiers in Physics, 12. https://doi.org/10.3389/fphy.2024.1411559

Vancouver

Zhang S, Xie M. MIPANet: optimizing RGB-D semantic segmentation through multi-modal interaction and pooling attention. Frontiers in Physics. 2024;12. doi:10.3389/fphy.2024.1411559

BibTeX
@article{Zhang2024,
  author = {Zhang, Shuai and Xie, Minghong},
  title = {MIPANet: optimizing RGB-D semantic segmentation through multi-modal interaction and pooling attention},
  journal = {Frontiers in Physics},
  year = {2024},
  volume = {12},
  doi = {10.3389/fphy.2024.1411559}
}
Kostenlos starten →