EUVIMEDEUVIMED entdecken
Crossref

A Closer Look at Conditional Prompt Tuning for Vision-Language Models

Ji Zhang, Shihan Wu, Lianli Gao, Jingkuan Song, Nicu Sebe, Heng Tao Shen

International Journal of Computer Vision · 2026-05 · 134 (5)

DOI: 10.1007/s11263-026-02759-6 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 06.04.2026. Verfasst von Ji Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 67 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite the great promise of Prompt Tuning (PT) in adapting large Vision-Language Pretrained Models (VLPMs) to downstream tasks, they often struggle to overcome the Base-New Tradeoff (BNT) dilemma: as VLPMs are better tuned to a base task, their ability to generalize to new tasks diminishes. Recent work on conditional PT addresses this problem by replacing static prompts with dynamic Visual Image Information (VII)-conditioned prompts, improving the model’s generalization to new tasks to some extent. In this work, we first identify a critical issue with existing conditional PT methods: using VII as the “condition” of prompts yields suboptimal performance, and even random noise-conditioned prompts can outperform the VII-conditioned counterparts. On further analysis, we find that learning dynamic prompts conditioned on Textual Class Information (TCI) is the key to solving the BNT problem. Motivated by this, we then propose Class-adaptive Prompt Tuning (CaPT), which enables fast adaptation of tuned models to new classes by learning TCI-conditioned prompts from base classes. Remarkably, CaPT can be used as a plugin to mitigate the BNT problem for existing unconditional PT schemes. Extensive experiments on 11 datasets show that CaPT consistently improves the performance of five strong unconditional PT baselines with negligible additional computational cost. Additionally, by integrating CaPT with our recently proposed DePT (Zhang et al., 2024) framework, we devise a new conditional PT approach, termed DeCaPT, which outperforms the H ACC of the state-of-the-art conditional PT scheme by 3.49%, averaged over the 11 datasets. Code: https://github.com/Koorye/CaPT.

Zitieren

APA

Zhang, J., Wu, S., Gao, L., Song, J., Sebe, N., & Shen, H. T. (2026). A Closer Look at Conditional Prompt Tuning for Vision-Language Models. International Journal of Computer Vision, 134(5). https://doi.org/10.1007/s11263-026-02759-6

Vancouver

Zhang J, Wu S, Gao L, Song J, Sebe N, Shen HT. A Closer Look at Conditional Prompt Tuning for Vision-Language Models. International Journal of Computer Vision. 2026;134(5). doi:10.1007/s11263-026-02759-6

BibTeX
@article{Zhang2026,
  author = {Zhang, Ji and Wu, Shihan and Gao, Lianli and Song, Jingkuan and Sebe, Nicu and Shen, Heng Tao},
  title = {A Closer Look at Conditional Prompt Tuning for Vision-Language Models},
  journal = {International Journal of Computer Vision},
  year = {2026},
  volume = {134},
  number = {5},
  doi = {10.1007/s11263-026-02759-6}
}
Kostenlos starten →