MOFM: A Multiple-in-One Flow Mamba for Unregistered Multi-Modal Image Fusion
IEEE Transactions on Circuits and Systems for Video Technology · 2026-08 · 36 (8) · 12296-12310
DOI: 10.1109/tcsvt.2026.3681540 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-08. Verfasst von Bo Yang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 62 Literaturangaben. Gefördert von: Science and Technology Innovation Program of Hunan Province, Natural Science Foundation for Young Scientists of Shanxi Province, Major Program of Xiangjiang Laboratory.
Abstract (Originalsprache)
Multi-modal image fusion aims to integrate complementary cues from different modalities into a single image, facilitating downstream tasks such as object detection. However, input image pairs are often misaligned due to rigid or non-rigid deformations in practical scenarios. Such misregistration introduces structural distortions and visual artifacts, reducing the reliability of the fused results and limiting their effectiveness for downstream vision applications. While existing methods demonstrate satisfactory results under specific deformation scenarios, they exhibit limited generalization to diverse and severe misregistrations. To this end, this study proposes a unified multiple-in-one flow Mamba framework for registering various image deformations and generating high-quality fused results. Specifically, a hierarchical flow Mamba is designed to model rigid and non-rigid flow fields and enhance adaptability to complex deformations by progressively refining misaligned features. To better distinguish between rigid and non-rigid types, a flow field classifier predicts rigid/non-rigid categories and provides prompts for high-level feature modulation. Furthermore, a flow-guided fusion Mamba module is developed to aggregate aligned multi-level modality features and generate fused images, while an iterative training strategy enables collaborative optimization by using fusion outputs to refine flow estimation. Experiments across three representative modality tasks demonstrate that the proposed method delivers superior fusion performance while maintaining applicability to object detection. The code will be available at: https://github.com/BOYang-pro/MOFM
Förderung
- Science and Technology Innovation Program of Hunan Province
- Natural Science Foundation for Young Scientists of Shanxi Province
- Major Program of Xiangjiang Laboratory
- Open Research Project of the State Key Laboratory of Industrial Control Technology
Zitieren
APA
Yang, B., Jiang, Z., Pan, D., Lin, Z., & Gui, W. (2026). MOFM: A Multiple-in-One Flow Mamba for Unregistered Multi-Modal Image Fusion. IEEE Transactions on Circuits and Systems for Video Technology, 36(8), 12296-12310. https://doi.org/10.1109/tcsvt.2026.3681540
Vancouver
Yang B, Jiang Z, Pan D, Lin Z, Gui W. MOFM: A Multiple-in-One Flow Mamba for Unregistered Multi-Modal Image Fusion. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(8):12296-12310. doi:10.1109/tcsvt.2026.3681540
BibTeX
@article{Yang2026,
author = {Yang, Bo and Jiang, Zhaohui and Pan, Dong and Lin, Zhiping and Gui, Weihua},
title = {MOFM: A Multiple-in-One Flow Mamba for Unregistered Multi-Modal Image Fusion},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {8},
pages = {12296--12310},
doi = {10.1109/tcsvt.2026.3681540}
}
EUVIMED