MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images
Sensors · 13.11.2024 · 24 (22) · 7266
DOI: 10.3390/s24227266 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 13.11.2024. Verfasst von Yan Wang und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Electrical and Electronic Engineering, Wuhan Polytechnic University, Wuhan 430023, China. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wuhan Polytechnic University.
Abstract (Originalsprache)
Semantic segmentation of remote sensing images is a fundamental task in computer vision, holding substantial relevance in applications such as land cover surveys, environmental protection, and urban building planning. In recent years, multi-modal fusion-based models have garnered considerable attention, exhibiting superior segmentation performance when compared with traditional single-modal techniques. Nonetheless, the majority of these multi-modal models, which rely on Convolutional Neural Networks (CNNs) or Vision Transformers (ViTs) for feature fusion, face limitations in terms of remote modeling capabilities or computational complexity. This paper presents a novel Mamba-based multi-modal fusion network called MFMamba for semantic segmentation of remote sensing images. Specifically, the network employs a dual-branch encoding structure, consisting of a CNN-based main encoder for extracting local features from high-resolution remote sensing images (HRRSIs) and of a Mamba-based auxiliary encoder for capturing global features on its corresponding digital surface model (DSM). To capitalize on the distinct attributes of the multi-modal remote sensing data from both branches, a feature fusion block (FFB) is designed to synergistically enhance and integrate the features extracted from the dual-branch structure at each stage. Extensive experiments on the Vaihingen and the Potsdam datasets have verified the effectiveness and superiority of MFMamba in semantic segmentation of remote sensing images. Compared with state-of-the-art methods, MFMamba achieves higher overall accuracy (OA) and a higher mean F1 score (mF1) and mean intersection over union (mIoU), while maintaining low computational complexity.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Electrical and Electronic Engineering, Wuhan Polytechnic University, Wuhan 430023, China.
- School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430081, China.
ORCID-Profile: Li Cao
Förderung
- Wuhan Polytechnic University
Zitieren
APA
Wang, Y., Cao, L., & Deng, H. (2024). MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images. Sensors, 24(22), 7266. https://doi.org/10.3390/s24227266
Vancouver
Wang Y, Cao L, Deng H. MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images. Sensors. 2024;24(22):7266. doi:10.3390/s24227266
BibTeX
@article{Wang2024,
author = {Wang, Yan and Cao, Li and Deng, He},
title = {MFMamba: A Mamba-Based Multi-Modal Fusion Network for Semantic Segmentation of Remote Sensing Images},
journal = {Sensors},
year = {2024},
volume = {24},
number = {22},
pages = {7266},
doi = {10.3390/s24227266}
}
EUVIMED