EUVIMEDEUVIMED entdecken
Crossref

Self-Adaptive Vision-Language Tracking With Context Prompting

Jie Zhao, Xin Chen, Shengming Li, Chunjuan Bo, Dong Wang, Huchuan Lu

IEEE Transactions on Image Processing · 2025 · 34 · 8046-8058

DOI: 10.1109/tip.2025.3635016 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Jie Zhao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 72 Literaturangaben. Gefördert von: National Natural Science Foundations of China, Liaoning Provincial Science and Technology Project, China Postdoctoral Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Due to the substantial gap between vision and language modalities, along with the mismatch problem between fixed language descriptions and dynamic visual information, existing vision-language tracking methods exhibit performance on par with or slightly worse than vision-only tracking. Effectively exploiting the rich semantics of language to enhance tracking robustness remains an open challenge. To address these issues, we propose a self-adaptive vision-language tracking framework that leverages the pre-trained multi-modal CLIP model to obtain well-aligned visual-language representations. A novel context-aware prompting mechanism is introduced to dynamically adapt linguistic cues based on the evolving visual context during tracking. Specifically, our context prompter extracts dynamic visual features from the current search image and integrates them into the text encoding process, enabling self-updating language embeddings. Furthermore, our framework employs a unified one-stream Transformer architecture, supporting joint training for both vision-only and vision-language tracking scenarios. Our method not only bridges the modality gap but also enhances robustness by allowing language features to evolve with visual context. Extensive experiments on four vision-language tracking benchmarks demonstrate that our method effectively leverages the advantages of language to enhance visual tracking. Our large model can obtain 55.0% AUC on $\text {LaSOT}_{\text {EXT}}$ and 69.0% AUC on TNL2K. Additionally, our language-only tracking model achieves performance comparable to that of state-of-the-art vision-only tracking methods on TNL2K. Code is available at https://github.com/zj5559/SAVLT.

Förderung

  • National Natural Science Foundations of China
  • Liaoning Provincial Science and Technology Project
  • China Postdoctoral Science Foundation
  • Funding of the Central Universities of China

Zitieren

APA

Zhao, J., Chen, X., Li, S., Bo, C., Wang, D., & Lu, H. (2025). Self-Adaptive Vision-Language Tracking With Context Prompting. IEEE Transactions on Image Processing, 34, 8046-8058. https://doi.org/10.1109/tip.2025.3635016

Vancouver

Zhao J, Chen X, Li S, Bo C, Wang D, Lu H. Self-Adaptive Vision-Language Tracking With Context Prompting. IEEE Transactions on Image Processing. 2025;34:8046-8058. doi:10.1109/tip.2025.3635016

BibTeX
@article{Zhao2025,
  author = {Zhao, Jie and Chen, Xin and Li, Shengming and Bo, Chunjuan and Wang, Dong and Lu, Huchuan},
  title = {Self-Adaptive Vision-Language Tracking With Context Prompting},
  journal = {IEEE Transactions on Image Processing},
  year = {2025},
  volume = {34},
  pages = {8046--8058},
  doi = {10.1109/tip.2025.3635016}
}
Kostenlos starten →