Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (4) · 3581-3589
DOI: 10.1609/aaai.v39i4.32372 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Xiantao Hu und 7 weiteren Autor:innen. Die Arbeit wurde bisher 47-mal zitiert (Crossref).
Abstract (Originalsprache)
Multimodal tracking has garnered widespread attention as a result of its ability to effectively address the inherent limitations of traditional RGB tracking. However, existing multimodal trackers mainly focus on the fusion and enhancement of spatial features or merely leverage the sparse temporal relationships between video frames. These approaches do not fully exploit the temporal correlations in multimodal videos, making it difficult to capture the dynamic changes and motion information of targets in complex scenarios. To alleviate this problem, we propose a unified multimodal spatial-temporal tracking approach named STTrack. In contrast to previous paradigms that solely relied on updating reference information, we introduced a temporal state generator (TSG) that continuously generates a sequence of tokens containing multimodal temporal information. These temporal information tokens are used to guide the localization of the target in the next time state, establish long-range contextual relationships between video frames, and capture the temporal trajectory of the target. Furthermore, at the spatial level, we introduced the mamba fusion and background suppression interactive (BSI) modules. These modules establish a dual-stage mechanism for coordinating information interaction and fusion between modalities. Extensive comparisons on five benchmark datasets illustrate that STTrack achieves state-of-the-art performance across various multimodal tracking scenarios.
Zitieren
APA
Hu, X., Tai, Y., Zhao, X., Zhao, C., Zhang, Z., Li, J., Zhong, B., & Yang, J. (2025). Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking. Proceedings of the AAAI Conference on Artificial Intelligence, 39(4), 3581-3589. https://doi.org/10.1609/aaai.v39i4.32372
Vancouver
Hu X, Tai Y, Zhao X, Zhao C, Zhang Z, Li J, et al. Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(4):3581-3589. doi:10.1609/aaai.v39i4.32372
BibTeX
@article{Hu2025,
author = {Hu, Xiantao and Tai, Ying and Zhao, Xu and Zhao, Chen and Zhang, Zhenyu and Li, Jun and Zhong, Bineng and Yang, Jian},
title = {Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {4},
pages = {3581--3589},
doi = {10.1609/aaai.v39i4.32372}
}
EUVIMED