EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CFANet: The Cross-Modal Fusion Attention Network for Indoor RGB-D Semantic Segmentation

Long-Fei Wu iD, Dan Wei iD, Chang-An Xu iD

Journal of Imaging · 27.05.2025 · 11 (6) · 177

DOI: 10.3390/jimaging11060177 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Imaging, online erschienen am 27.05.2025. Verfasst von Long-Fei Wu und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Mechanical and Automotive Engineering, Shanghai University of Engineering Science, Shanghai 201620, China. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Indoor image semantic segmentation technology is applied to fields such as smart homes and indoor security. The challenges faced by semantic segmentation techniques using RGB images and depth maps as data sources include the semantic gap between RGB images and depth maps and the loss of detailed information. To address these issues, a multi-head self-attention mechanism is adopted to adaptively align features of the two modalities and perform feature fusion in both spatial and channel dimensions. Appropriate feature extraction methods are designed according to the different characteristics of RGB images and depth maps. For RGB images, asymmetric convolution is introduced to capture features in the horizontal and vertical directions, enhance short-range information dependence, mitigate the gridding effect of dilated convolution, and introduce criss-cross attention to obtain contextual information from global dependency relationships. On the depth map, a strategy of extracting significant unimodal features from the channel and spatial dimensions is used. A lightweight skip connection module is designed to fuse low-level and high-level features. In addition, since the first layer contains the richest detailed information and the last layer contains rich semantic information, a feature refinement head is designed to fuse the two. The method achieves an mIoU of 53.86% and 51.85% on the NYUDv2 and SUN-RGBD datasets, which is superior to mainstream methods.

Schlagwörter

Stichwörter der Autor:innen

RGB-Dcross-modal fusionfeature extractionfeature interaction

Autor:innen und Institutionen

  • School of Mechanical and Automotive Engineering, Shanghai University of Engineering Science, Shanghai 201620, China.
  • College of Materials and Energy, South China Agricultural University, Guangzhou 510642, China.

ORCID-Profile: Long-Fei Wu, Dan Wei, Chang-An Xu

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Wu, L. F., Wei, D., & Xu, C. A. (2025). CFANet: The Cross-Modal Fusion Attention Network for Indoor RGB-D Semantic Segmentation. Journal of Imaging, 11(6), 177. https://doi.org/10.3390/jimaging11060177

Vancouver

Wu LF, Wei D, Xu CA. CFANet: The Cross-Modal Fusion Attention Network for Indoor RGB-D Semantic Segmentation. Journal of Imaging. 2025;11(6):177. doi:10.3390/jimaging11060177

BibTeX
@article{Wu2025,
  author = {Wu, Long-Fei and Wei, Dan and Xu, Chang-An},
  title = {CFANet: The Cross-Modal Fusion Attention Network for Indoor RGB-D Semantic Segmentation},
  journal = {Journal of Imaging},
  year = {2025},
  volume = {11},
  number = {6},
  pages = {177},
  doi = {10.3390/jimaging11060177}
}
Kostenlos starten →