EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization

Qiangxing Tian, Min Zhang

Entropy · 14.03.2025 · 27 (3) · 301

DOI: 10.3390/e27030301 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Entropy, online erschienen am 14.03.2025. Verfasst von Qiangxing Tian und 1 weiteren Autor:innen. Beteiligte Einrichtung: School of Information and Electrical Engineering, Hangzhou City University, Hangzhou 310015, China. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Prompt tuning visual-language models (VLMs) for specialized tasks often involves leveraging task-specific textual tokens, which can tailor the pre-existing, broad capabilities of a VLM to more narrowly focused applications. This approach, exemplified by CoOp-based methods, integrates mutable textual tokens with categorical tokens to foster nuanced textual comprehension. Nonetheless, such specialized textual insights often fail to generalize beyond the scope of familiar categories, as they tend to overshadow the versatile, general textual knowledge intrinsic to the model’s wide-ranging applicability. Addressing this base-novel dilemma, we propose the innovative concept of SparseKnowledge-guided Context Optimization (Sparse-KgCoOp). This technique aims to fortify the adaptable prompts’ capacity to generalize to categories yet unencountered. The cornerstone of Sparse-KgCoOp is based on the premise that reducing the differences between adaptive prompt and their hand-crafted counterparts through sparsification operations can mitigate the erosion of fundamental knowledge. Specifically, Sparse-KgCoOp seeks to narrow the gap between the textual embeddings produced by both the dynamic prompts and the manually devised ones, thus preserving the foundational knowledge while maintaining adaptability. Extensive experiments of several benchmarks demonstrate that the proposed Sparse-KgCoOp is an efficient method for prompt tuning.

Schlagwörter

Stichwörter der Autor:innen

prompt tuningsparse knowledge-guided context optimizationvisual-language models

Autor:innen und Institutionen

  • School of Information and Electrical Engineering, Hangzhou City University, Hangzhou 310015, China.
  • School of Computer Science and Technology, East China Normal University, Shanghai 200062, China.

Zitieren

APA

Tian, Q., & Zhang, M. (2025). Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization. Entropy, 27(3), 301. https://doi.org/10.3390/e27030301

Vancouver

Tian Q, Zhang M. Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization. Entropy. 2025;27(3):301. doi:10.3390/e27030301

BibTeX
@article{Tian2025,
  author = {Tian, Qiangxing and Zhang, Min},
  title = {Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization},
  journal = {Entropy},
  year = {2025},
  volume = {27},
  number = {3},
  pages = {301},
  doi = {10.3390/e27030301}
}
Kostenlos starten →