EUVIMEDEUVIMED entdecken
Crossref

TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network

Dongyu Rao, Tianyang Xu, Xiao-Jun Wu

IEEE Transactions on Image Processing · 2024 · 1-1

DOI: 10.1109/tip.2023.3273451 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2024. Verfasst von Dongyu Rao und 2 weiteren Autor:innen. Die Arbeit wurde bisher 172-mal zitiert (Crossref). Gefördert von: National Natural Science Foundation of China, 111 Project of Ministry of Education of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The end-to-end image fusion framework has achieved promising performance, with dedicated convolutional networks aggregating the multi-modal local appearance. However, long-range dependencies are directly neglected in existing CNN fusion approaches, impeding balancing the entire image-level perception for complex scenario fusion. In this paper, therefore, we propose an infrared and visible image fusion algorithm based on the transformer module and adversarial learning. Inspired by the global interaction power, we use the transformer technique to learn the effective global fusion relations. In particular, shallow features extracted by CNN are interacted in the proposed transformer fusion module to refine the fusion relationship within the spatial scope and across channels simultaneously. Besides, adversarial learning is designed in the training process to improve the output discrimination via imposing competitive consistency from the inputs, reflecting the specific characteristics in infrared and visible images. The experimental performance demonstrates the effectiveness of the proposed modules, with superior improvement against the state-of-the-art, generalising a novel paradigm via transformer and adversarial learning in the fusion task.

Förderung

  • National Natural Science Foundation of China
  • 111 Project of Ministry of Education of China

Zitieren

APA

Rao, D., Xu, T., & Wu, X. J. (2024). TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network. IEEE Transactions on Image Processing, 1-1. https://doi.org/10.1109/tip.2023.3273451

Vancouver

Rao D, Xu T, Wu XJ. TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network. IEEE Transactions on Image Processing. 2024:1-1. doi:10.1109/tip.2023.3273451

BibTeX
@article{Rao2024,
  author = {Rao, Dongyu and Xu, Tianyang and Wu, Xiao-Jun},
  title = {TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network},
  journal = {IEEE Transactions on Image Processing},
  year = {2024},
  pages = {1--1},
  doi = {10.1109/tip.2023.3273451}
}
Kostenlos starten →