EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

HATF: Multi-Modal Feature Learning for Infrared and Visible Image Fusion via Hybrid Attention Transformer

Xiangzeng Liu, Ziyao Wang, Haojie Gao, Xiang Li, Lei Wang, Qiguang Miao

Remote Sensing · 25.02.2024 · 16 (5) · 803

DOI: 10.3390/rs16050803 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 25.02.2024. Verfasst von Xiangzeng Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Basic Research Program of Shaanxi, Aeronautical Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Current CNN-based methods for infrared and visible image fusion are limited by the low discrimination of extracted structural features, the adoption of uniform loss functions, and the lack of inter-modal feature interaction, which make it difficult to obtain optimal fusion results. To alleviate the above problems, a framework for multimodal feature learning fusion using a cross-attention Transformer is proposed. To extract rich structural features at different scales, residual U-Nets with mixed receptive fields are adopted to capture salient object information at various granularities. Then, a hybrid attention fusion strategy is employed to integrate the complementing information from the input images. Finally, adaptive loss functions are designed to achieve optimal fusion results for different modal features. The fusion framework proposed in this study is thoroughly evaluated using the TNO, FLIR, and LLVIP datasets, encompassing diverse scenes and varying illumination conditions. In the comparative experiments, HATF achieved competitive results on three datasets, with EN, SD, MI, and SSIM metrics reaching the best performance on the TNO dataset, surpassing the second-best method by 2.3%, 18.8%, 4.2%, and 2.2%, respectively. These results validate the effectiveness of the proposed method in terms of both robustness and image fusion quality compared to several popular methods.

Förderung

  • Natural Science Basic Research Program of Shaanxi
  • Aeronautical Science Foundation of China

Zitieren

APA

Liu, X., Wang, Z., Gao, H., Li, X., Wang, L., & Miao, Q. (2024). HATF: Multi-Modal Feature Learning for Infrared and Visible Image Fusion via Hybrid Attention Transformer. Remote Sensing, 16(5), 803. https://doi.org/10.3390/rs16050803

Vancouver

Liu X, Wang Z, Gao H, Li X, Wang L, Miao Q. HATF: Multi-Modal Feature Learning for Infrared and Visible Image Fusion via Hybrid Attention Transformer. Remote Sensing. 2024;16(5):803. doi:10.3390/rs16050803

BibTeX
@article{Liu2024,
  author = {Liu, Xiangzeng and Wang, Ziyao and Gao, Haojie and Li, Xiang and Wang, Lei and Miao, Qiguang},
  title = {HATF: Multi-Modal Feature Learning for Infrared and Visible Image Fusion via Hybrid Attention Transformer},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {5},
  pages = {803},
  doi = {10.3390/rs16050803}
}
Kostenlos starten →