Infrared and Visible Image Fusion via Lightweight Semantic Prior Encoding and Cross-Attention Fusion
Sensors · 06.07.2026 · 26 (13) · 4300
DOI: 10.3390/s26134300 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 06.07.2026. Verfasst von Xun Zhang und 3 weiteren Autor:innen. Beteiligte Einrichtung: College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Shandong Province.
Abstract (Originalsprache)
Infrared (IR) and visible image fusion aims to synthesize a composite representation that integrates the thermal target saliency of IR imagery with the textural richness of visible imagery. Existing deep learning-based methods have achieved promising progress in this field. However, they either operate at the pixel level without semantic priors, or rely on segmentation supervision to obtain such priors. Both approaches limit their practicality and performance in complex scenes. To design a lightweight fusion network that leverages semantic priors without segmentation supervision, we propose SPE2Fusion, a semantic prior-driven fusion network that operates through a dual-stage semantic injection paradigm. Specifically, a lightweight semantic encoder is designed to extract multi-scale scene priors in an end-to-end manner optimized solely by the fusion loss, without requiring segmentation mask annotations. Then, these priors are injected at two complementary stages: the Efficient Semantic Feature Awareness (ESFA) module applies spatially adaptive attention at the encoding stage to amplify semantically salient regions, while the Efficient Semantic Feature Embedding (ESFE) module applies semantically conditioned spatial normalization at the decoding stage to ensure coherent texture reconstruction. Finally, a bidirectional cross-attention fusion block is introduced to integrate complementary cross-modal features under this dual semantic guidance. The network is supervised by a multi-constraint loss combining gradient fidelity, intensity preservation, and structural similarity terms. Comprehensive experiments on the MSRS, LLVIP, and RoadScene benchmarks demonstrate that SPE2Fusion achieves state-of-the-art performance against representative methods (e.g., CrossFuse and DDBFusion), ranking first on four of six metrics on the MSRS test set, specifically EN (6.70), QAB/F (0.86), AG (6.06), and SD (43.44), while maintaining strong generalization on unseen datasets without domain adaptation.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China.
- Qingdao Innovation and Development Center, Harbin Engineering University, Qingdao 266000, China.
ORCID-Profile: Xun Zhang
Förderung
- Natural Science Foundation of Shandong Province
Zitieren
APA
Zhang, X., Wu, D., Li, J., & Cui, N. (2026). Infrared and Visible Image Fusion via Lightweight Semantic Prior Encoding and Cross-Attention Fusion. Sensors, 26(13), 4300. https://doi.org/10.3390/s26134300
Vancouver
Zhang X, Wu D, Li J, Cui N. Infrared and Visible Image Fusion via Lightweight Semantic Prior Encoding and Cross-Attention Fusion. Sensors. 2026;26(13):4300. doi:10.3390/s26134300
BibTeX
@article{Zhang2026,
author = {Zhang, Xun and Wu, Di and Li, Jianqi and Cui, Na},
title = {Infrared and Visible Image Fusion via Lightweight Semantic Prior Encoding and Cross-Attention Fusion},
journal = {Sensors},
year = {2026},
volume = {26},
number = {13},
pages = {4300},
doi = {10.3390/s26134300}
}
EUVIMED