EUVIMEDEUVIMED entdecken
Crossref

DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models

Guiqin Wang, Peng Zhao, Xiang Wang, Haoran Guo, Nan Qi, Shusen Yang, Qinghai Guo

IEEE Transactions on Circuits and Systems for Video Technology · 2026-06 · 36 (6) · 8590-8601

DOI: 10.1109/tcsvt.2026.3657756 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-06. Verfasst von Guiqin Wang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 67 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Test-Time Adaptation (TTA) has recently emerged as a promising research direction, enabling vision-language models (VLMs) to adapt to unlabeled test data in zero-shot settings. Among TTA approaches, test-time prompt tuning has shown great potential for enhancing the practical applicability of VLMs. However, existing methods typically either focus on adapting a single modality or apply uniform optimization to both modalities, without explicitly defining modality-specific optimization objectives. Such a one-size-fits-all strategy often results in suboptimal performance under test-time conditions. To address this limitation, we propose Dual-modality Heterogeneous Prompt Tuning (DHPT), a novel framework designed to simultaneously capture fine-grained textual semantics and alleviate domain shift noise in the visual modality. Specifically, we leverage a large language model to provide textual cognition guidance for the text encoder, while on the vision side, we develop a lightweight calibration module that adaptively mitigates domain shift noise across different scales. Furthermore, we introduce a cluster-tight optimization objective that enhances the stability and generalizability of prompt tuning under distribution shifts. Extensive experiments conducted on 11 benchmark datasets demonstrate that DHPT consistently and significantly outperforms existing TTA methods for VLMs.

Zitieren

APA

Wang, G., Zhao, P., Wang, X., Guo, H., Qi, N., Yang, S., & Guo, Q. (2026). DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models. IEEE Transactions on Circuits and Systems for Video Technology, 36(6), 8590-8601. https://doi.org/10.1109/tcsvt.2026.3657756

Vancouver

Wang G, Zhao P, Wang X, Guo H, Qi N, Yang S, et al. DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(6):8590-8601. doi:10.1109/tcsvt.2026.3657756

BibTeX
@article{Wang2026,
  author = {Wang, Guiqin and Zhao, Peng and Wang, Xiang and Guo, Haoran and Qi, Nan and Yang, Shusen and Guo, Qinghai},
  title = {DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2026},
  volume = {36},
  number = {6},
  pages = {8590--8601},
  doi = {10.1109/tcsvt.2026.3657756}
}
Kostenlos starten →