EUVIMEDEUVIMED entdecken
Crossref

Unifying Visual and Vision-Language Tracking via Contrastive Learning

Yinchao Ma, Yuyang Tang, Wenfei Yang, Tianzhu Zhang, Jinpeng Zhang, Mengxue Kang

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4107-4116

DOI: 10.1609/aaai.v38i5.28205 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yinchao Ma und 5 weiteren Autor:innen. Die Arbeit wurde bisher 64-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Single object tracking aims to locate the target object in a video sequence according to the state specified by different modal references, including the initial bounding box (BBOX), natural language (NL), or both (NL+BBOX). Due to the gap between different modalities, most existing trackers are designed for single or partial of these reference settings and overspecialize on the specific modality. Differently, we present a unified tracker called UVLTrack, which can simultaneously handle all three reference settings (BBOX, NL, NL+BBOX) with the same parameters. The proposed UVLTrack enjoys several merits. First, we design a modality-unified feature extractor for joint visual and language feature learning and propose a multi-modal contrastive loss to align the visual and language features into a unified semantic space. Second, a modality-adaptive box head is proposed, which makes full use of the target reference to mine ever-changing scenario features dynamically from video contexts and distinguish the target in a contrastive way, enabling robust performance in different reference settings. Extensive experimental results demonstrate that UVLTrack achieves promising performance on seven visual tracking datasets, three vision-language tracking datasets, and three visual grounding datasets. Codes and models will be open-sourced at https://github.com/OpenSpaceAI/UVLTrack.

Zitieren

APA

Ma, Y., Tang, Y., Yang, W., Zhang, T., Zhang, J., & Kang, M. (2024). Unifying Visual and Vision-Language Tracking via Contrastive Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4107-4116. https://doi.org/10.1609/aaai.v38i5.28205

Vancouver

Ma Y, Tang Y, Yang W, Zhang T, Zhang J, Kang M. Unifying Visual and Vision-Language Tracking via Contrastive Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4107-4116. doi:10.1609/aaai.v38i5.28205

BibTeX
@article{Ma2024,
  author = {Ma, Yinchao and Tang, Yuyang and Yang, Wenfei and Zhang, Tianzhu and Zhang, Jinpeng and Kang, Mengxue},
  title = {Unifying Visual and Vision-Language Tracking via Contrastive Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {5},
  pages = {4107--4116},
  doi = {10.1609/aaai.v38i5.28205}
}
Kostenlos starten →