Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization
Entropy · 14.03.2025 · 27 (3) · 301
DOI: 10.3390/e27030301 ↗Kurz gefasst
Fachartikel in der Zeitschrift Entropy, online erschienen am 14.03.2025. Verfasst von Qiangxing Tian und 1 weiteren Autor:innen. Beteiligte Einrichtung: School of Information and Electrical Engineering, Hangzhou City University, Hangzhou 310015, China. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Prompt tuning visual-language models (VLMs) for specialized tasks often involves leveraging task-specific textual tokens, which can tailor the pre-existing, broad capabilities of a VLM to more narrowly focused applications. This approach, exemplified by CoOp-based methods, integrates mutable textual tokens with categorical tokens to foster nuanced textual comprehension. Nonetheless, such specialized textual insights often fail to generalize beyond the scope of familiar categories, as they tend to overshadow the versatile, general textual knowledge intrinsic to the model’s wide-ranging applicability. Addressing this base-novel dilemma, we propose the innovative concept of SparseKnowledge-guided Context Optimization (Sparse-KgCoOp). This technique aims to fortify the adaptable prompts’ capacity to generalize to categories yet unencountered. The cornerstone of Sparse-KgCoOp is based on the premise that reducing the differences between adaptive prompt and their hand-crafted counterparts through sparsification operations can mitigate the erosion of fundamental knowledge. Specifically, Sparse-KgCoOp seeks to narrow the gap between the textual embeddings produced by both the dynamic prompts and the manually devised ones, thus preserving the foundational knowledge while maintaining adaptability. Extensive experiments of several benchmarks demonstrate that the proposed Sparse-KgCoOp is an efficient method for prompt tuning.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Information and Electrical Engineering, Hangzhou City University, Hangzhou 310015, China.
- School of Computer Science and Technology, East China Normal University, Shanghai 200062, China.
Zitieren
APA
Tian, Q., & Zhang, M. (2025). Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization. Entropy, 27(3), 301. https://doi.org/10.3390/e27030301
Vancouver
Tian Q, Zhang M. Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization. Entropy. 2025;27(3):301. doi:10.3390/e27030301
BibTeX
@article{Tian2025,
author = {Tian, Qiangxing and Zhang, Min},
title = {Enhancing Visual-Language Prompt Tuning Through Sparse Knowledge-Guided Context Optimization},
journal = {Entropy},
year = {2025},
volume = {27},
number = {3},
pages = {301},
doi = {10.3390/e27030301}
}
EUVIMED