Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination
IEEE Transactions on Image Processing · 2025 · 34 · 7943-7958
DOI: 10.1109/tip.2025.3635048 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Guanghui Yue und 6 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: Guangdong Basic and Applied Basic Research Foundation, Shenzhen Medical Research Fund, Project of Department of Education of Guangdong Province.
Abstract (Originalsprache)
Although text-guided infrared-visible image fusion helps improve content understanding under extreme illumination, existing methods usually ignore semantic differences between textual and visual features, resulting in limited improvement. To address this challenge, we propose a Text-Guided Semantic Alignment Network, termed TSANet, for extreme-illumination infrared-visible image fusion. The network follows an encoder-decoder structure, with two image encoders, two text encoders, and one decoder. It uses a Semantic Alignment and Fusion (SAF) block to bridge the two image encoders in each layer. Specifically, the SAF block consists of two parallel Semantic Alignment (SA) modules, corresponding to the infrared and visible modalities, respectively, and a Spatial-Frequency Interaction (SFI) module. The SA module aligns the visual feature from the image encoder with its corresponding textual feature from the text encoder, to guide the network focus on key semantic regions of infrared and visible images. The SFI module aggregates the spatial and frequency information extracted from the modality-aligned features of two SA modules for complementary representation learning. The network progressively complements two image modalities by connecting the SAF blocks from top to down, and finally provides a visually pleasing fusion effect by feeding the output of the last block into the decoder. Recognizing that existing datasets lack illumination diversity, we contribute a new dataset specifically designed for extreme-illumination image fusion. Extensive experiments show the effectiveness and superiority of TSANet over seven state-of-the-art methods. The source code and dataset are available at https://github.com/WentaoLi-CV/TSANet.
Förderung
- Guangdong Basic and Applied Basic Research Foundation
- Shenzhen Medical Research Fund
- Project of Department of Education of Guangdong Province
- National Natural Science Foundation of China
- Natural Science Foundation of Shenzhen
- Medicine Plus Program of Shenzhen University
- China Postdoctoral Science Foundation
- "Leading Goos" Research and Development Program of Zhejiang Province
Zitieren
APA
Yue, G., Li, W., Zhao, C., Wu, Z., Zhou, T., Jiang, Q., & Cong, R. (2025). Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination. IEEE Transactions on Image Processing, 34, 7943-7958. https://doi.org/10.1109/tip.2025.3635048
Vancouver
Yue G, Li W, Zhao C, Wu Z, Zhou T, Jiang Q, et al. Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination. IEEE Transactions on Image Processing. 2025;34:7943-7958. doi:10.1109/tip.2025.3635048
BibTeX
@article{Yue2025,
author = {Yue, Guanghui and Li, Wentao and Zhao, Cheng and Wu, Zhiliang and Zhou, Tianwei and Jiang, Qiuping and Cong, Runmin},
title = {Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination},
journal = {IEEE Transactions on Image Processing},
year = {2025},
volume = {34},
pages = {7943--7958},
doi = {10.1109/tip.2025.3635048}
}
EUVIMED