EUVIMEDEUVIMED entdecken
Crossref

Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion

Hui Sun, Long Lv, Pingping Zhang, Tongdan Tang, Feng Tian, Weibing Sun, Huchuan Lu

IEEE Transactions on Image Processing · 2025 · 34 · 7684-7696

DOI: 10.1109/tip.2025.3632221 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Hui Sun und 6 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Dalian Science and Technology Innovation Fund, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-Modal Image Fusion (MMIF) aims to integrate complementary image information from different modalities to produce informative images. Previous deep learning-based MMIF methods generally adopt Convolutional Neural Networks (CNNs) or Transformers for feature extraction. However, these methods deliver unsatisfactory performances due to the limited receptive field of CNNs and the high computational cost of Transformers. Recently, Mamba has demonstrated a powerful potential for modeling long-range dependencies with linear complexity, providing a promising solution to MMIF. Unfortunately, Mamba lacks full spatial and frequency perceptions, which are very important for MMIF. Moreover, employing Image Reconstruction (IR) as an auxiliary task has been proven beneficial for MMIF. However, a primary challenge is how to leverage IR efficiently and effectively. To address the above issues, we propose a novel framework named Spatial-Frequency Enhanced Mamba Fusion (SFMFusion) for MMIF. More specifically, we first propose a three-branch structure to couple MMIF and IR, which can retain complete contents from source images. Then, we propose the Spatial-Frequency Enhanced Mamba Block (SFMB), which can enhance Mamba in both spatial and frequency domains for comprehensive feature extraction. Finally, we propose the Dynamic Fusion Mamba Block (DFMB), which can be deployed across different branches for dynamic feature fusion. Extensive experiments show that our method achieves better results than most state-of-the-art methods on six MMIF datasets. The source code is available at https://github.com/SunHui1216/SFMFusion.

Förderung

  • National Natural Science Foundation of China
  • Dalian Science and Technology Innovation Fund
  • Fundamental Research Funds for the Central Universities
  • Natural Science Foundation of Liaoning Province

Zitieren

APA

Sun, H., Lv, L., Zhang, P., Tang, T., Tian, F., Sun, W., & Lu, H. (2025). Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion. IEEE Transactions on Image Processing, 34, 7684-7696. https://doi.org/10.1109/tip.2025.3632221

Vancouver

Sun H, Lv L, Zhang P, Tang T, Tian F, Sun W, et al. Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion. IEEE Transactions on Image Processing. 2025;34:7684-7696. doi:10.1109/tip.2025.3632221

BibTeX
@article{Sun2025,
  author = {Sun, Hui and Lv, Long and Zhang, Pingping and Tang, Tongdan and Tian, Feng and Sun, Weibing and Lu, Huchuan},
  title = {Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion},
  journal = {IEEE Transactions on Image Processing},
  year = {2025},
  volume = {34},
  pages = {7684--7696},
  doi = {10.1109/tip.2025.3632221}
}
Kostenlos starten →