Unifying Visual and Vision-Language Tracking via Contrastive Learning
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4107-4116
DOI: 10.1609/aaai.v38i5.28205 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yinchao Ma und 5 weiteren Autor:innen. Die Arbeit wurde bisher 64-mal zitiert (Crossref).
Abstract (Originalsprache)
Single object tracking aims to locate the target object in a video sequence according to the state specified by different modal references, including the initial bounding box (BBOX), natural language (NL), or both (NL+BBOX). Due to the gap between different modalities, most existing trackers are designed for single or partial of these reference settings and overspecialize on the specific modality. Differently, we present a unified tracker called UVLTrack, which can simultaneously handle all three reference settings (BBOX, NL, NL+BBOX) with the same parameters. The proposed UVLTrack enjoys several merits. First, we design a modality-unified feature extractor for joint visual and language feature learning and propose a multi-modal contrastive loss to align the visual and language features into a unified semantic space. Second, a modality-adaptive box head is proposed, which makes full use of the target reference to mine ever-changing scenario features dynamically from video contexts and distinguish the target in a contrastive way, enabling robust performance in different reference settings. Extensive experimental results demonstrate that UVLTrack achieves promising performance on seven visual tracking datasets, three vision-language tracking datasets, and three visual grounding datasets. Codes and models will be open-sourced at https://github.com/OpenSpaceAI/UVLTrack.
Zitieren
APA
Ma, Y., Tang, Y., Yang, W., Zhang, T., Zhang, J., & Kang, M. (2024). Unifying Visual and Vision-Language Tracking via Contrastive Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4107-4116. https://doi.org/10.1609/aaai.v38i5.28205
Vancouver
Ma Y, Tang Y, Yang W, Zhang T, Zhang J, Kang M. Unifying Visual and Vision-Language Tracking via Contrastive Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4107-4116. doi:10.1609/aaai.v38i5.28205
BibTeX
@article{Ma2024,
author = {Ma, Yinchao and Tang, Yuyang and Yang, Wenfei and Zhang, Tianzhu and Zhang, Jinpeng and Kang, Mengxue},
title = {Unifying Visual and Vision-Language Tracking via Contrastive Learning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {5},
pages = {4107--4116},
doi = {10.1609/aaai.v38i5.28205}
}
EUVIMED