TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network
IEEE Transactions on Image Processing · 2024 · 1-1
DOI: 10.1109/tip.2023.3273451 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2024. Verfasst von Dongyu Rao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 172-mal zitiert (Crossref). Gefördert von: National Natural Science Foundation of China, 111 Project of Ministry of Education of China.
Abstract (Originalsprache)
The end-to-end image fusion framework has achieved promising performance, with dedicated convolutional networks aggregating the multi-modal local appearance. However, long-range dependencies are directly neglected in existing CNN fusion approaches, impeding balancing the entire image-level perception for complex scenario fusion. In this paper, therefore, we propose an infrared and visible image fusion algorithm based on the transformer module and adversarial learning. Inspired by the global interaction power, we use the transformer technique to learn the effective global fusion relations. In particular, shallow features extracted by CNN are interacted in the proposed transformer fusion module to refine the fusion relationship within the spatial scope and across channels simultaneously. Besides, adversarial learning is designed in the training process to improve the output discrimination via imposing competitive consistency from the inputs, reflecting the specific characteristics in infrared and visible images. The experimental performance demonstrates the effectiveness of the proposed modules, with superior improvement against the state-of-the-art, generalising a novel paradigm via transformer and adversarial learning in the fusion task.
Förderung
- National Natural Science Foundation of China
- 111 Project of Ministry of Education of China
Zitieren
APA
Rao, D., Xu, T., & Wu, X. J. (2024). TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network. IEEE Transactions on Image Processing, 1-1. https://doi.org/10.1109/tip.2023.3273451
Vancouver
Rao D, Xu T, Wu XJ. TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network. IEEE Transactions on Image Processing. 2024:1-1. doi:10.1109/tip.2023.3273451
BibTeX
@article{Rao2024,
author = {Rao, Dongyu and Xu, Tianyang and Wu, Xiao-Jun},
title = {TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network},
journal = {IEEE Transactions on Image Processing},
year = {2024},
pages = {1--1},
doi = {10.1109/tip.2023.3273451}
}
EUVIMED