Bidirectional Interaction of CNN and Transformer Feature for Visual Tracking
IEEE Transactions on Circuits and Systems for Video Technology · 2024-08 · 34 (8) · 7259-7271
DOI: 10.1109/tcsvt.2024.3376690 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-08. Verfasst von Baozhen Sun und 4 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 92 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Empowered by the sophisticated long-range dependency modeling ability of Transformer, tracking performance has seen a dynamic increase in recent years. Approaches in this vein leverage the Transformer feature to integrate the information of target and search regions while neglecting the superior local representation extracted by their CNN backbone. To address this, we introduce a BIdirectional inTeraction mechanism between CNN and Transformer features for visual tracking, termed BIT-Tracker, which admits a comprehensive fusion of local and global representations, and thus boosts tracking performance. The first ingredient of BIT-Tracker is an aggregation of multi-level Transformer features to achieve a better global modeling ability. In order to combine the merits of both local and global representations, our second ingredient performs a bi-directional interaction between CNN and Transformer features, where the interaction is achieved via either querying the CNN feature from the Transformer feature or querying the Transformer feature from the CNN feature. Afterwards, the outputs from both directions are fused to predict the temporal locations of targets. Extensive experiments demonstrate the effectiveness of the proposed feature aggregation and bi-directional interaction modules. Impressively, BIT-Tracker achieves leading performance on eight tracking benchmarks and outperforms SOTA results by salient margins. Code will be made available.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Sun, B., Wang, Z., Wang, S., Cheng, Y., & Ning, J. (2024). Bidirectional Interaction of CNN and Transformer Feature for Visual Tracking. IEEE Transactions on Circuits and Systems for Video Technology, 34(8), 7259-7271. https://doi.org/10.1109/tcsvt.2024.3376690
Vancouver
Sun B, Wang Z, Wang S, Cheng Y, Ning J. Bidirectional Interaction of CNN and Transformer Feature for Visual Tracking. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(8):7259-7271. doi:10.1109/tcsvt.2024.3376690
BibTeX
@article{Sun2024,
author = {Sun, Baozhen and Wang, Zhenhua and Wang, Shilei and Cheng, Yongkang and Ning, Jifeng},
title = {Bidirectional Interaction of CNN and Transformer Feature for Visual Tracking},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {8},
pages = {7259--7271},
doi = {10.1109/tcsvt.2024.3376690}
}
EUVIMED