EUVIMEDEUVIMED entdecken
Crossref

MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion

Jia Lei, Jiawei Li, Jinyuan Liu, Bin Wang, Shihua Zhou, Qiang Zhang, Xiaopeng Wei, Nikola K. Kasabov

IEEE Transactions on Multimedia · 2025 · 27 · 3880-3894

DOI: 10.1109/tmm.2025.3535355 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Multimedia, 2025. Verfasst von Jia Lei und 7 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: Higher Education Discipline Innovation Project, National Natural Science Foundation of China, Postgraduate Education Reform Project of Liaoning Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modality image fusion (MMIF) entails synthesizing images with detailed textures and prominent objects. Existing methods tend to use general feature extraction to handle different fusion tasks. However, these methods have difficulty breaking fusion barriers across various modalities owing to the lack of targeted learning routes. In this work, we propose a multi-scenario feature joint learning architecture, MLFuse, that employs the commonalities of multi-modality images to deconstruct the fusion progress. Specifically, we construct a cross-modal knowledge reinforcing network that adopts a multipath calibration strategy to promote information communication between different images. In addition, two professional networks are developed to maintain the salient and textural information of fusion results. The spatial-spectral domain optimizing network can learn the vital relationship of the source image context with the help of spatial attention and spectral attention. The edge-guided learning network utilizes the convolution operations of various receptive fields to capture image texture information. The desired fusion results are obtained by aggregating the outputs from the three networks. Extensive experiments demonstrate the superiority of MLFuse for infrared-visible image fusion and medical image fusion. The excellent results of downstream tasks (i.e., object detection and semantic segmentation) further verify the high-quality fusion performance of our method.

Förderung

  • Higher Education Discipline Innovation Project
  • National Natural Science Foundation of China
  • Postgraduate Education Reform Project of Liaoning Province
  • Artificial Intelligence Innovation Development Plan Project of Liaoning Province
  • Dalian Outstanding Young Science and Technology Talent Support Program
  • Dalian Major Projects of Basic Research
  • Interdisciplinary Project of Dalian University

Zitieren

APA

Lei, J., Li, J., Liu, J., Wang, B., Zhou, S., Zhang, Q., Wei, X., & Kasabov, N. K. (2025). MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion. IEEE Transactions on Multimedia, 27, 3880-3894. https://doi.org/10.1109/tmm.2025.3535355

Vancouver

Lei J, Li J, Liu J, Wang B, Zhou S, Zhang Q, et al. MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion. IEEE Transactions on Multimedia. 2025;27:3880-3894. doi:10.1109/tmm.2025.3535355

BibTeX
@article{Lei2025,
  author = {Lei, Jia and Li, Jiawei and Liu, Jinyuan and Wang, Bin and Zhou, Shihua and Zhang, Qiang and Wei, Xiaopeng and Kasabov, Nikola K.},
  title = {MLFuse: Multi-Scenario Feature Joint Learning for Multi-Modality Image Fusion},
  journal = {IEEE Transactions on Multimedia},
  year = {2025},
  volume = {27},
  pages = {3880--3894},
  doi = {10.1109/tmm.2025.3535355}
}
Kostenlos starten →