EUVIMEDEUVIMED entdecken
Crossref

Generalizing Vision-Language Models with Dedicated Prompt Guidance

Xinyao Li, Yinjie Min, Hongbo Chen, Zhekai Du, Fengling Li, Jingjing Li

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (28) · 23239-23247

DOI: 10.1609/aaai.v40i28.39492 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xinyao Li und 5 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Fine-tuning large pretrained vision-language models (VLMs) has emerged as a prevalent paradigm for downstream adaptation, yet it faces a critical trade-off between domain specificity and domain generalization (DG) ability. Current methods typically fine-tune a universal model on the entire dataset, which potentially compromises the ability to generalize to unseen domains. To fill this gap, we provide a theoretical understanding of the generalization ability for VLM fine-tuning, which reveals that training multiple parameter-efficient expert models on partitioned source domains leads to better generalization than fine-tuning a universal model. Inspired by this finding, we propose a two-step domain-expert-Guided DG (GuiDG) framework. GuiDG first employs prompt tuning to obtain source domain experts, then introduces a Cross-Modal Attention module to guide the fine-tuning of the vision encoder via adaptive expert integration. To better evaluate few-shot DG, we construct ImageNet-DG from ImageNet and its variants. Extensive experiments on standard DG benchmarks and ImageNet-DG demonstrate that GuiDG improves upon state-of-the-art fine-tuning methods while maintaining efficiency.

Zitieren

APA

Li, X., Min, Y., Chen, H., Du, Z., Li, F., & Li, J. (2026). Generalizing Vision-Language Models with Dedicated Prompt Guidance. Proceedings of the AAAI Conference on Artificial Intelligence, 40(28), 23239-23247. https://doi.org/10.1609/aaai.v40i28.39492

Vancouver

Li X, Min Y, Chen H, Du Z, Li F, Li J. Generalizing Vision-Language Models with Dedicated Prompt Guidance. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(28):23239-23247. doi:10.1609/aaai.v40i28.39492

BibTeX
@article{Li2026,
  author = {Li, Xinyao and Min, Yinjie and Chen, Hongbo and Du, Zhekai and Li, Fengling and Li, Jingjing},
  title = {Generalizing Vision-Language Models with Dedicated Prompt Guidance},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {28},
  pages = {23239--23247},
  doi = {10.1609/aaai.v40i28.39492}
}
Kostenlos starten →