EUVIMEDEUVIMED entdecken
Crossref

RGBT Tracking via Progressive Fusion Transformer With Dynamically Guided Learning

Yabin Zhu, Chenglong Li, Xiao Wang, Jin Tang, Zhixiang Huang

IEEE Transactions on Circuits and Systems for Video Technology · 2024-09 · 34 (9) · 8722-8735

DOI: 10.1109/tcsvt.2024.3391802 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-09. Verfasst von Yabin Zhu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 70 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Anhui Province, Natural Science Foundation of Anhui Higher Education Institution.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing Transformer-based RGB-Thermal (RGBT) tracking methods either use cross-attention to fuse the two modalities, or use self-attention and cross-attention to model both modality-specific and modality-sharing information. However, the significant appearance gap between modalities limits the feature representation ability of certain modalities during the fusion process. To address this problem, we propose a novel Progressive Fusion Transformer called ProFormer, which progressively integrates single-modality information into the multimodal representation for robust RGBT tracking. In particular, ProFormer first uses a self-attention module to collaboratively extract the multimodal representation. Then, ProFormer introduces two cross-attention modules to interact it with the features of the dual modalities for enhancing modality-specific information in the multimodal representation. In addition, we propose a dynamically guided learning algorithm that adaptively employs the well-performing branches to guide the learning of other branches, to improve the representation ability of each branch. Extensive experiments demonstrate that our proposed ProFormer achieves a new state-of-the-art performance on RGBT210, RGBT234, LasHeR, and VTUAV datasets.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Anhui Province
  • Natural Science Foundation of Anhui Higher Education Institution
  • Research Funds of Center for Big Data and Population Health of Institute of Health and Medcine

Zitieren

APA

Zhu, Y., Li, C., Wang, X., Tang, J., & Huang, Z. (2024). RGBT Tracking via Progressive Fusion Transformer With Dynamically Guided Learning. IEEE Transactions on Circuits and Systems for Video Technology, 34(9), 8722-8735. https://doi.org/10.1109/tcsvt.2024.3391802

Vancouver

Zhu Y, Li C, Wang X, Tang J, Huang Z. RGBT Tracking via Progressive Fusion Transformer With Dynamically Guided Learning. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(9):8722-8735. doi:10.1109/tcsvt.2024.3391802

BibTeX
@article{Zhu2024,
  author = {Zhu, Yabin and Li, Chenglong and Wang, Xiao and Tang, Jin and Huang, Zhixiang},
  title = {RGBT Tracking via Progressive Fusion Transformer With Dynamically Guided Learning},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2024},
  volume = {34},
  number = {9},
  pages = {8722--8735},
  doi = {10.1109/tcsvt.2024.3391802}
}
Kostenlos starten →