One-Stream Stepwise Decreasing for Vision-Language Tracking
IEEE Transactions on Circuits and Systems for Video Technology · 2024-10 · 34 (10) · 9053-9063
DOI: 10.1109/tcsvt.2024.3395352 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-10. Verfasst von Guangtong Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 38 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Project of Guangxi Science and Technology, Guangxi “Young Bagui Scholar” Teams for Innovation and Research Project.
Abstract (Originalsprache)
Based on the fixed language descriptions in the initial frames, a vision-language tracker typically adopts a two-stream model structure to align vision and language features at the feature fusion stages. However, this paradigm may degrade the tracking performance due to inaccurate language descriptions and lacks further modal interaction. To address these issues, we propose a one-stream vision-language model called One-stream Stepwise Decreasing for Vision-Language Tracking (OSDT). Specifically, we first encode the language description using a language encoder. The obtained language features are then combined with visual images and entered jointly into a visual encoder, in which the encoder’s self-attention mechanism is utilized to facilitate more interactions between language and visual features. Moreover, to mitigate the problems caused by inaccurate language descriptions, we design a stepwise decreasing multi-modal interaction framework, in which a Feature Filter Module (FFM) is introduced to select language features that are more relevant to visual information to provide semantic guidance for visual feature extraction. Furthermore, without additional feature fusion modules, our one-stream model framework can efficiently utilize the proposed feature filtering module for feature selection. Consequently, our tracker can achieve fast tracking speed in the vision-language tracking domain compared to existing state-of-the-art methods. We extensively evaluate our tracker on three benchmarks, i.e. TNL2K, LaSOT, and OTB99, demonstrating competing performance compared to state-of-the-art vision-language tracking methods.
Förderung
- National Natural Science Foundation of China
- Project of Guangxi Science and Technology
- Guangxi “Young Bagui Scholar” Teams for Innovation and Research Project
- Guangxi “Bagui Scholar” Teams for Innovation and Research Project
- Guangxi Collaborative Innovation Center of Multi-Source Information Integration and Intelligent Processing
- Guangxi Talent Highland Project of Big Data Intelligence and Application
- Research Project of Guangxi Normal University
Zitieren
APA
Zhang, G., Zhong, B., Liang, Q., Mo, Z., Li, N., & Song, S. (2024). One-Stream Stepwise Decreasing for Vision-Language Tracking. IEEE Transactions on Circuits and Systems for Video Technology, 34(10), 9053-9063. https://doi.org/10.1109/tcsvt.2024.3395352
Vancouver
Zhang G, Zhong B, Liang Q, Mo Z, Li N, Song S. One-Stream Stepwise Decreasing for Vision-Language Tracking. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(10):9053-9063. doi:10.1109/tcsvt.2024.3395352
BibTeX
@article{Zhang2024,
author = {Zhang, Guangtong and Zhong, Bineng and Liang, Qihua and Mo, Zhiyi and Li, Ning and Song, Shuxiang},
title = {One-Stream Stepwise Decreasing for Vision-Language Tracking},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {10},
pages = {9053--9063},
doi = {10.1109/tcsvt.2024.3395352}
}
EUVIMED