EUVIMEDEUVIMED entdecken
Crossref

Plug-and-play Class-aware Knowledge Injection for Prompt Learning with Visual-Language Model

Junhui Yin, Nan Pu, Xinyu Zhang, Lingfeng Yang, Lin Wu, Xiaojie Wang, Zhun Zhong

International Journal of Computer Vision · 2026-06 · 134 (6)

DOI: 10.1007/s11263-026-02851-x ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 02.05.2026. Verfasst von Junhui Yin und 6 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Prompt learning has become an effective and widely used technique in enhancing vision-language models (VLMs) such as CLIP for various downstream tasks, particularly in zero-shot classification within specific domains. Existing methods typically focus on either learning class-shared prompts for a given domain or generating instance-specific prompts through conditional prompt learning. While these methods have achieved promising performance, they often overlook class-specific knowledge in prompt design, leading to suboptimal outcomes. The underlying reasons are: 1) class-specific prompts offer more fine-grained supervision compared to coarse class-shared prompts, which helps prevent misclassification of data from different classes into a single class; 2) compared to class-specific prompts, instance-specific prompts neglect the richer class-level information across multiple instances, potentially causing data from the same class to be divided into multiple classes. To effectively supplement the class-specific knowledge into existing methods, we propose a plug-and-play Class-Aware Knowledge Injection (CAKI) framework. CAKI comprises two key components, i.e., class-specific prompt generation and query-key prompt matching. The former encodes class-specific knowledge into prompts from few-shot samples that belong to the same class and stores the learned prompts in a class-level knowledge bank. The latter provides a plug-and-play mechanism for each test instance to retrieve relevant class-level knowledge from the knowledge bank and inject such knowledge to refine model predictions. Extensive experiments demonstrate that our CAKI effectively improves the performance of existing methods on base and novel classes. Code is publicly available at https://github.com/yjh576/CAKI.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Yin, J., Pu, N., Zhang, X., Yang, L., Wu, L., Wang, X., & Zhong, Z. (2026). Plug-and-play Class-aware Knowledge Injection for Prompt Learning with Visual-Language Model. International Journal of Computer Vision, 134(6). https://doi.org/10.1007/s11263-026-02851-x

Vancouver

Yin J, Pu N, Zhang X, Yang L, Wu L, Wang X, et al. Plug-and-play Class-aware Knowledge Injection for Prompt Learning with Visual-Language Model. International Journal of Computer Vision. 2026;134(6). doi:10.1007/s11263-026-02851-x

BibTeX
@article{Yin2026,
  author = {Yin, Junhui and Pu, Nan and Zhang, Xinyu and Yang, Lingfeng and Wu, Lin and Wang, Xiaojie and Zhong, Zhun},
  title = {Plug-and-play Class-aware Knowledge Injection for Prompt Learning with Visual-Language Model},
  journal = {International Journal of Computer Vision},
  year = {2026},
  volume = {134},
  number = {6},
  doi = {10.1007/s11263-026-02851-x}
}
Kostenlos starten →