Generalizing Vision-Language Models with Dedicated Prompt Guidance
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (28) · 23239-23247
DOI: 10.1609/aaai.v40i28.39492 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xinyao Li und 5 weiteren Autor:innen.
Abstract (Originalsprache)
Fine-tuning large pretrained vision-language models (VLMs) has emerged as a prevalent paradigm for downstream adaptation, yet it faces a critical trade-off between domain specificity and domain generalization (DG) ability. Current methods typically fine-tune a universal model on the entire dataset, which potentially compromises the ability to generalize to unseen domains. To fill this gap, we provide a theoretical understanding of the generalization ability for VLM fine-tuning, which reveals that training multiple parameter-efficient expert models on partitioned source domains leads to better generalization than fine-tuning a universal model. Inspired by this finding, we propose a two-step domain-expert-Guided DG (GuiDG) framework. GuiDG first employs prompt tuning to obtain source domain experts, then introduces a Cross-Modal Attention module to guide the fine-tuning of the vision encoder via adaptive expert integration. To better evaluate few-shot DG, we construct ImageNet-DG from ImageNet and its variants. Extensive experiments on standard DG benchmarks and ImageNet-DG demonstrate that GuiDG improves upon state-of-the-art fine-tuning methods while maintaining efficiency.
Zitieren
APA
Li, X., Min, Y., Chen, H., Du, Z., Li, F., & Li, J. (2026). Generalizing Vision-Language Models with Dedicated Prompt Guidance. Proceedings of the AAAI Conference on Artificial Intelligence, 40(28), 23239-23247. https://doi.org/10.1609/aaai.v40i28.39492
Vancouver
Li X, Min Y, Chen H, Du Z, Li F, Li J. Generalizing Vision-Language Models with Dedicated Prompt Guidance. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(28):23239-23247. doi:10.1609/aaai.v40i28.39492
BibTeX
@article{Li2026,
author = {Li, Xinyao and Min, Yinjie and Chen, Hongbo and Du, Zhekai and Li, Fengling and Li, Jingjing},
title = {Generalizing Vision-Language Models with Dedicated Prompt Guidance},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {28},
pages = {23239--23247},
doi = {10.1609/aaai.v40i28.39492}
}
EUVIMED