Adaptive cross-modal alignment via symmetric prompt tuning for few-shot vision–language learning
Journal of King Saud University Computer and Information Sciences · 2026-08 · 38 (6)
DOI: 10.1007/s44443-026-00952-8 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of King Saud University Computer and Information Sciences, online erschienen am 09.07.2026. Verfasst von Feidu Akmel und 4 weiteren Autor:innen. Die Arbeit nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Fundamental Research Funds for the Central Universities, National Natural Science Foundation of China, Sichuan Science and Technology Program.
Abstract (Originalsprache)
Few-shot learning with vision-language models suffers from a fundamental structural limitation: support and query samples are processed through independent and asymmetric encoding pipelines. This causes query-side semantic blindness, where the model lacks rich cross-modal interactions during query encoding. Consequently, it weakens vision-language alignment and creates a training-inference distribution gap, degrading generalization to novel categories. Existing prompt-based methods inherit this asymmetry and thus cannot leverage text-conditioned semantic context on the query side at inference time. To address this limitation, we propose an Adaptive Cross-Modal Alignment via Symmetric Prompt Tuning for Few-Shot Vision–Language Learning (ACAS-PT) a unified framework that resolves this issue via symmetric prompt tuning. ACAS-PT applies identical prompt-guided, text-conditioned feature transformations to both support and query samples in a shared multimodal space, eliminating the distribution gap by design. Specifically, we propose two modules. First, a Semantic-Aware Class-Embedding Learner transforms prompt-conditioned CLIP class embeddings into class-specific semantic vectors used to modulate both support and query visual features via FiLM-based affine transformation, ensuring that query samples receive the same class-specific semantic grounding as support prototypes at inference. Second, an Adaptive Similarity Guided Module (ASGM) replaces fragile equal-weight prototype averaging with learnable instance-weighted centroid aggregation and a per-class-pair cross-modal alignment matrix that gates classification scores by within-class semantic-visual alignment confidence, yielding robust prototype estimates even under extreme label scarcity. Extensive experiments on four benchmark datasets show ACAS-PT outperforms 16 state-of-the-art methods, with symmetric processing alone yielding up to a +2.5% improvement in 5-shot accuracy. These results highlight query-side semantic blindness as a critical bottleneck in vision-language few-shot learning.
Förderung
- Fundamental Research Funds for the Central Universities
- National Natural Science Foundation of China
- Sichuan Science and Technology Program
Zitieren
APA
Akmel, F., Gong, X., Hadabi, A., Naouri, A., & Ibrahim, A. O. (2026). Adaptive cross-modal alignment via symmetric prompt tuning for few-shot vision–language learning. Journal of King Saud University Computer and Information Sciences, 38(6). https://doi.org/10.1007/s44443-026-00952-8
Vancouver
Akmel F, Gong X, Hadabi A, Naouri A, Ibrahim AO. Adaptive cross-modal alignment via symmetric prompt tuning for few-shot vision–language learning. Journal of King Saud University Computer and Information Sciences. 2026;38(6). doi:10.1007/s44443-026-00952-8
BibTeX
@article{Akmel2026,
author = {Akmel, Feidu and Gong, Xun and Hadabi, Abdalla and Naouri, Abdenacer and Ibrahim, Ashraf Osman},
title = {Adaptive cross-modal alignment via symmetric prompt tuning for few-shot vision–language learning},
journal = {Journal of King Saud University Computer and Information Sciences},
year = {2026},
volume = {38},
number = {6},
doi = {10.1007/s44443-026-00952-8}
}
EUVIMED