MSDF-Mamba: Mutual-Spectrum Perception Deformable Fusion Mamba for Drone-Based Visible–Infrared Cross-Modality Vehicle Detection
Remote Sensing · 15.12.2025 · 17 (24) · 4037
DOI: 10.3390/rs17244037 ↗Kurz gefasst
Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 15.12.2025. Verfasst von Jiashuo Shen und 5 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Natural Science Foundation of China.
Abstract (Originalsprache)
To ensure all-day detection performance, unmanned aerial vehicles (UAVs) usually need both visible and infrared images for dual-modality fusion object detection. However, misalignment between the RGB-IR image pairs and complexity of fusion models constrain the fusion detection performance. Specifically, typical alignment methods choose only one modality as a reference modality, leading to excessive dependence on the chosen modality quality. Furthermore, current multimodal fusion detection methods still struggle to strike a balance between high accuracy and low computational complexity, thus making the deployment of these models on resource-constrained UAV platforms a challenge. In order to solve the above problems, this paper proposes a dual-modality UAV image target detection method named Mutual-Spectrum Perception Deformable Fusion Mamba (MSDF-Mamba). First, we designed a Mutual Spectral Deformable Alignment (MSDA) module. This module employs a bidirectional cross-attention mechanism to enable one modality to actively extract the semantic information of the other, generating fusion features rich in cross-modal context as shared references. These fusion features are then used to predict spatial offsets, with deformable convolutions achieving feature alignment. Based on the MSDA module, a Selective Scan Fusion (SSF) module is carefully designed to project the aligned features onto a unified hidden state space. With this method, we achieve full interaction and enhanced fusion of intermodal features with low computational complexity. Experiment results demonstrate that our method outperforms existing state-of-the-art cross-modality detection methods on the mAP metric, achieving a relative improvement of 3.1% compared to baseline models such as DMM, while still maintaining high computational efficiency.
Förderung
- the National Natural Science Foundation of China
Zitieren
APA
Shen, J., He, J., Liu, Q., Zhang, Z., Wang, G., & Lu, D. (2025). MSDF-Mamba: Mutual-Spectrum Perception Deformable Fusion Mamba for Drone-Based Visible–Infrared Cross-Modality Vehicle Detection. Remote Sensing, 17(24), 4037. https://doi.org/10.3390/rs17244037
Vancouver
Shen J, He J, Liu Q, Zhang Z, Wang G, Lu D. MSDF-Mamba: Mutual-Spectrum Perception Deformable Fusion Mamba for Drone-Based Visible–Infrared Cross-Modality Vehicle Detection. Remote Sensing. 2025;17(24):4037. doi:10.3390/rs17244037
BibTeX
@article{Shen2025,
author = {Shen, Jiashuo and He, Jun and Liu, Qiuyu and Zhang, Zhilong and Wang, Guoyan and Lu, Dawei},
title = {MSDF-Mamba: Mutual-Spectrum Perception Deformable Fusion Mamba for Drone-Based Visible–Infrared Cross-Modality Vehicle Detection},
journal = {Remote Sensing},
year = {2025},
volume = {17},
number = {24},
pages = {4037},
doi = {10.3390/rs17244037}
}
EUVIMED