TUFusion: A Transformer-Based Universal Fusion Algorithm for Multimodal Images
IEEE Transactions on Circuits and Systems for Video Technology · 2024-03 · 34 (3) · 1712-1725
DOI: 10.1109/tcsvt.2023.3296745 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-03. Verfasst von Yangyang Zhao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 137-mal zitiert (Crossref) und nennt 52 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Multimodal image fusion is one of the important research directions in the field of multimodal fusion. This technique can realize image and data enhancement by using complementary multimodal images and be widely used in medicine, industry, security and fire protection, automatic driving and consumer electronics. In this work, we propose a transformer-based universal fusion (TUFusion) algorithm, and it has a multidomain fusion capability. The advantage of TUFusion algorithm is the design of hybrid transformer and convolutional neural network (CNN) encoder structure and a new composite attention fusion strategy, which has the ability of global and local information integration. Compared with the classical state-of-the-art multimodal image fusion methods, the experimental result on multidomain data sets showed that the TUFusion algorithm has certain universality in image fusion. Meanwhile, the TUFusion algorithm we proposed achieves good values on peak signal to noise ratio (PSNR), root mean square error (RMSE) and structural similarity index measure (SSIM). The code of the TUFusion algorithm in this article is available at https://github.com/windrunners/TUFusion.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Zhao, Y., Zheng, Q., Zhu, P., Zhang, X., & Ma, W. (2024). TUFusion: A Transformer-Based Universal Fusion Algorithm for Multimodal Images. IEEE Transactions on Circuits and Systems for Video Technology, 34(3), 1712-1725. https://doi.org/10.1109/tcsvt.2023.3296745
Vancouver
Zhao Y, Zheng Q, Zhu P, Zhang X, Ma W. TUFusion: A Transformer-Based Universal Fusion Algorithm for Multimodal Images. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(3):1712-1725. doi:10.1109/tcsvt.2023.3296745
BibTeX
@article{Zhao2024,
author = {Zhao, Yangyang and Zheng, Qingchun and Zhu, Peihao and Zhang, Xu and Ma, Wenpeng},
title = {TUFusion: A Transformer-Based Universal Fusion Algorithm for Multimodal Images},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {3},
pages = {1712--1725},
doi = {10.1109/tcsvt.2023.3296745}
}
EUVIMED