A Novel Mamba Architecture with a Semantic Transformer for Efficient Real-Time Remote Sensing Semantic Segmentation
Remote Sensing · 17.07.2024 · 16 (14) · 2620
DOI: 10.3390/rs16142620 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 17.07.2024. Verfasst von Hao Ding und 6 weiteren Autor:innen. Die Arbeit wurde bisher 69-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Key R&D Program of Shandong Province of China, National Key Research and Development Program of China, Key Research and Development Program of Jiangsu Province.
Abstract (Originalsprache)
Real-time remote sensing segmentation technology is crucial for unmanned aerial vehicles (UAVs) in battlefield surveillance, land characterization observation, earthquake disaster assessment, etc., and can significantly enhance the application value of UAVs in military and civilian fields. To realize this potential, it is essential to develop real-time semantic segmentation methods that can be applied to resource-limited platforms, such as edge devices. The majority of mainstream real-time semantic segmentation methods rely on convolutional neural networks (CNNs) and transformers. However, CNNs cannot effectively capture long-range dependencies, while transformers have high computational complexity. This paper proposes a novel remote sensing Mamba architecture for real-time segmentation tasks in remote sensing, named RTMamba. Specifically, the backbone utilizes a Visual State-Space (VSS) block to extract deep features and maintains linear computational complexity, thereby capturing long-range contextual information. Additionally, a novel Inverted Triangle Pyramid Pooling (ITP) module is incorporated into the decoder. The ITP module can effectively filter redundant feature information and enhance the perception of objects and their boundaries in remote sensing images. Extensive experiments were conducted on three challenging aerial remote sensing segmentation benchmarks, including Vaihingen, Potsdam, and LoveDA. The results show that RTMamba achieves competitive performance advantages in terms of segmentation accuracy and inference speed compared to state-of-the-art CNN and transformer methods. To further validate the deployment potential of the model on embedded devices with limited resources, such as UAVs, we conducted tests on the Jetson AGX Orin edge device. The experimental results demonstrate that RTMamba achieves impressive real-time segmentation performance.
Förderung
- Key R&D Program of Shandong Province of China
- National Key Research and Development Program of China
- Key Research and Development Program of Jiangsu Province
- Distinguished Young Scholar of Shandong Province
- Taishan Scholars Program
- Major Basic Research Projects of Shandong Province
Zitieren
APA
Ding, H., Xia, B., Liu, W., Zhang, Z., Zhang, J., Wang, X., & Xu, S. (2024). A Novel Mamba Architecture with a Semantic Transformer for Efficient Real-Time Remote Sensing Semantic Segmentation. Remote Sensing, 16(14), 2620. https://doi.org/10.3390/rs16142620
Vancouver
Ding H, Xia B, Liu W, Zhang Z, Zhang J, Wang X, et al. A Novel Mamba Architecture with a Semantic Transformer for Efficient Real-Time Remote Sensing Semantic Segmentation. Remote Sensing. 2024;16(14):2620. doi:10.3390/rs16142620
BibTeX
@article{Ding2024,
author = {Ding, Hao and Xia, Bo and Liu, Weilin and Zhang, Zekai and Zhang, Jinglin and Wang, Xing and Xu, Sen},
title = {A Novel Mamba Architecture with a Semantic Transformer for Efficient Real-Time Remote Sensing Semantic Segmentation},
journal = {Remote Sensing},
year = {2024},
volume = {16},
number = {14},
pages = {2620},
doi = {10.3390/rs16142620}
}
EUVIMED