EUVIMEDEUVIMED entdecken
Crossref

Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination

Guanghui Yue, Wentao Li, Cheng Zhao, Zhiliang Wu, Tianwei Zhou, Qiuping Jiang, Runmin Cong

IEEE Transactions on Image Processing · 2025 · 34 · 7943-7958

DOI: 10.1109/tip.2025.3635048 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Guanghui Yue und 6 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: Guangdong Basic and Applied Basic Research Foundation, Shenzhen Medical Research Fund, Project of Department of Education of Guangdong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Although text-guided infrared-visible image fusion helps improve content understanding under extreme illumination, existing methods usually ignore semantic differences between textual and visual features, resulting in limited improvement. To address this challenge, we propose a Text-Guided Semantic Alignment Network, termed TSANet, for extreme-illumination infrared-visible image fusion. The network follows an encoder-decoder structure, with two image encoders, two text encoders, and one decoder. It uses a Semantic Alignment and Fusion (SAF) block to bridge the two image encoders in each layer. Specifically, the SAF block consists of two parallel Semantic Alignment (SA) modules, corresponding to the infrared and visible modalities, respectively, and a Spatial-Frequency Interaction (SFI) module. The SA module aligns the visual feature from the image encoder with its corresponding textual feature from the text encoder, to guide the network focus on key semantic regions of infrared and visible images. The SFI module aggregates the spatial and frequency information extracted from the modality-aligned features of two SA modules for complementary representation learning. The network progressively complements two image modalities by connecting the SAF blocks from top to down, and finally provides a visually pleasing fusion effect by feeding the output of the last block into the decoder. Recognizing that existing datasets lack illumination diversity, we contribute a new dataset specifically designed for extreme-illumination image fusion. Extensive experiments show the effectiveness and superiority of TSANet over seven state-of-the-art methods. The source code and dataset are available at https://github.com/WentaoLi-CV/TSANet.

Förderung

  • Guangdong Basic and Applied Basic Research Foundation
  • Shenzhen Medical Research Fund
  • Project of Department of Education of Guangdong Province
  • National Natural Science Foundation of China
  • Natural Science Foundation of Shenzhen
  • Medicine Plus Program of Shenzhen University
  • China Postdoctoral Science Foundation
  • "Leading Goos" Research and Development Program of Zhejiang Province

Zitieren

APA

Yue, G., Li, W., Zhao, C., Wu, Z., Zhou, T., Jiang, Q., & Cong, R. (2025). Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination. IEEE Transactions on Image Processing, 34, 7943-7958. https://doi.org/10.1109/tip.2025.3635048

Vancouver

Yue G, Li W, Zhao C, Wu Z, Zhou T, Jiang Q, et al. Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination. IEEE Transactions on Image Processing. 2025;34:7943-7958. doi:10.1109/tip.2025.3635048

BibTeX
@article{Yue2025,
  author = {Yue, Guanghui and Li, Wentao and Zhao, Cheng and Wu, Zhiliang and Zhou, Tianwei and Jiang, Qiuping and Cong, Runmin},
  title = {Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination},
  journal = {IEEE Transactions on Image Processing},
  year = {2025},
  volume = {34},
  pages = {7943--7958},
  doi = {10.1109/tip.2025.3635048}
}
Kostenlos starten →