MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion
IEEE Transactions on Multimedia · 2025 · 27 · 3880-3894
DOI: 10.1109/tmm.2025.3535355 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Multimedia, 2025. Verfasst von Jia Lei und 7 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: Higher Education Discipline Innovation Project, National Natural Science Foundation of China, Postgraduate Education Reform Project of Liaoning Province.
Abstract (Originalsprache)
Multi-modality image fusion (MMIF) entails synthesizing images with detailed textures and prominent objects. Existing methods tend to use general feature extraction to handle different fusion tasks. However, these methods have difficulty breaking fusion barriers across various modalities owing to the lack of targeted learning routes. In this work, we propose a multi-scenario feature joint learning architecture, MLFuse, that employs the commonalities of multi-modality images to deconstruct the fusion progress. Specifically, we construct a cross-modal knowledge reinforcing network that adopts a multipath calibration strategy to promote information communication between different images. In addition, two professional networks are developed to maintain the salient and textural information of fusion results. The spatial-spectral domain optimizing network can learn the vital relationship of the source image context with the help of spatial attention and spectral attention. The edge-guided learning network utilizes the convolution operations of various receptive fields to capture image texture information. The desired fusion results are obtained by aggregating the outputs from the three networks. Extensive experiments demonstrate the superiority of MLFuse for infrared-visible image fusion and medical image fusion. The excellent results of downstream tasks (i.e., object detection and semantic segmentation) further verify the high-quality fusion performance of our method.
Förderung
- Higher Education Discipline Innovation Project
- National Natural Science Foundation of China
- Postgraduate Education Reform Project of Liaoning Province
- Artificial Intelligence Innovation Development Plan Project of Liaoning Province
- Dalian Outstanding Young Science and Technology Talent Support Program
- Dalian Major Projects of Basic Research
- Interdisciplinary Project of Dalian University
Zitieren
APA
Lei, J., Li, J., Liu, J., Wang, B., Zhou, S., Zhang, Q., Wei, X., & Kasabov, N. K. (2025). MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion. IEEE Transactions on Multimedia, 27, 3880-3894. https://doi.org/10.1109/tmm.2025.3535355
Vancouver
Lei J, Li J, Liu J, Wang B, Zhou S, Zhang Q, et al. MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion. IEEE Transactions on Multimedia. 2025;27:3880-3894. doi:10.1109/tmm.2025.3535355
BibTeX
@article{Lei2025,
author = {Lei, Jia and Li, Jiawei and Liu, Jinyuan and Wang, Bin and Zhou, Shihua and Zhang, Qiang and Wei, Xiaopeng and Kasabov, Nikola K.},
title = {MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion},
journal = {IEEE Transactions on Multimedia},
year = {2025},
volume = {27},
pages = {3880--3894},
doi = {10.1109/tmm.2025.3535355}
}
EUVIMED