EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images

Yan Wang, Li Cao iD, He Deng

Sensors · 13.11.2024 · 24 (22) · 7266

DOI: 10.3390/s24227266 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 13.11.2024. Verfasst von Yan Wang und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Electrical and Electronic Engineering, Wuhan Polytechnic University, Wuhan 430023, China. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wuhan Polytechnic University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Semantic segmentation of remote sensing images is a fundamental task in computer vision, holding substantial relevance in applications such as land cover surveys, environmental protection, and urban building planning. In recent years, multi-modal fusion-based models have garnered considerable attention, exhibiting superior segmentation performance when compared with traditional single-modal techniques. Nonetheless, the majority of these multi-modal models, which rely on Convolutional Neural Networks (CNNs) or Vision Transformers (ViTs) for feature fusion, face limitations in terms of remote modeling capabilities or computational complexity. This paper presents a novel Mamba-based multi-modal fusion network called MFMamba for semantic segmentation of remote sensing images. Specifically, the network employs a dual-branch encoding structure, consisting of a CNN-based main encoder for extracting local features from high-resolution remote sensing images (HRRSIs) and of a Mamba-based auxiliary encoder for capturing global features on its corresponding digital surface model (DSM). To capitalize on the distinct attributes of the multi-modal remote sensing data from both branches, a feature fusion block (FFB) is designed to synergistically enhance and integrate the features extracted from the dual-branch structure at each stage. Extensive experiments on the Vaihingen and the Potsdam datasets have verified the effectiveness and superiority of MFMamba in semantic segmentation of remote sensing images. Compared with state-of-the-art methods, MFMamba achieves higher overall accuracy (OA) and a higher mean F1 score (mF1) and mean intersection over union (mIoU), while maintaining low computational complexity.

Schlagwörter

Stichwörter der Autor:innen

feature fusionmulti-modal remote sensing datasemantic segmentation

Autor:innen und Institutionen

  • School of Electrical and Electronic Engineering, Wuhan Polytechnic University, Wuhan 430023, China.
  • School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430081, China.

ORCID-Profile: Li Cao

Förderung

  • Wuhan Polytechnic University

Zitieren

APA

Wang, Y., Cao, L., & Deng, H. (2024). MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images. Sensors, 24(22), 7266. https://doi.org/10.3390/s24227266

Vancouver

Wang Y, Cao L, Deng H. MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images. Sensors. 2024;24(22):7266. doi:10.3390/s24227266

BibTeX
@article{Wang2024,
  author = {Wang, Yan and Cao, Li and Deng, He},
  title = {MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {22},
  pages = {7266},
  doi = {10.3390/s24227266}
}
Kostenlos starten →