EUVIMEDEUVIMED entdecken
Crossref

Vision-Language Tracking With CLIP and Interactive Prompt Learning

Hong Zhu, Qingyang Lu, Lei Xue, Pingping Zhang, Guanglin Yuan

IEEE Transactions on Intelligent Transportation Systems · 2025-03 · 26 (3) · 3659-3670

DOI: 10.1109/tits.2024.3520103 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Intelligent Transportation Systems, 2025-03. Verfasst von Hong Zhu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 52 Literaturangaben. Gefördert von: Natural Science Foundation of Anhui Province, Scientific Research Project of National University of Defense Technology, Foundation of Army Artillery and Air Defense Academy of PLA.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision-language tracking is a new rising topic in intelligent transportation systems, particularly significant in autonomous driving and road surveillance. It is a task that aims to combine visual and auxiliary linguistic modalities to co-locate the target object in a video sequence. Currently, multi-modal data scarcity and burdensome modality fusion have become two major factors in limiting the development of vision-language tracking. To tackle the issues, we propose an efficient and effective one-stage vision-language tracking framework (CPIPTrack) that unifies feature extraction and multi-modal fusion by interactive prompt learning. Feature extraction is performed by the high-performance vision-language foundation model CLIP, resulting in the impressive generalization ability inherited from the large-scale model. Modality fusion is simplified to a few lightweight prompts, leading to significant savings in computational resources. Specifically, we design three types of prompts to dynamically learn the layer-wise feature relationships between vision and language, facilitating rich context interactions while enabling the pre-trained CLIP adaptation. In this manner, discriminative target-oriented visual features can be extracted by language and template guidance, which are used for subsequent reasoning. Due to the elimination of extra heavy modality fusion, the proposed CPIPTrack shows high efficiency in both training and inference. CPIPTrack has been extensively evaluated on three benchmark datasets, and the experimental results demonstrate that it achieves a good performance-speed balance with an AUC of 66.0% on LaSOT and a runtime of 51.7 FPS on RTX2080 Super.

Förderung

  • Natural Science Foundation of Anhui Province
  • Scientific Research Project of National University of Defense Technology
  • Foundation of Army Artillery and Air Defense Academy of PLA

Zitieren

APA

Zhu, H., Lu, Q., Xue, L., Zhang, P., & Yuan, G. (2025). Vision-Language Tracking With CLIP and Interactive Prompt Learning. IEEE Transactions on Intelligent Transportation Systems, 26(3), 3659-3670. https://doi.org/10.1109/tits.2024.3520103

Vancouver

Zhu H, Lu Q, Xue L, Zhang P, Yuan G. Vision-Language Tracking With CLIP and Interactive Prompt Learning. IEEE Transactions on Intelligent Transportation Systems. 2025;26(3):3659-3670. doi:10.1109/tits.2024.3520103

BibTeX
@article{Zhu2025,
  author = {Zhu, Hong and Lu, Qingyang and Xue, Lei and Zhang, Pingping and Yuan, Guanglin},
  title = {Vision-Language Tracking With CLIP and Interactive Prompt Learning},
  journal = {IEEE Transactions on Intelligent Transportation Systems},
  year = {2025},
  volume = {26},
  number = {3},
  pages = {3659--3670},
  doi = {10.1109/tits.2024.3520103}
}
Kostenlos starten →