DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models
IEEE Transactions on Circuits and Systems for Video Technology · 2026-06 · 36 (6) · 8590-8601
DOI: 10.1109/tcsvt.2026.3657756 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-06. Verfasst von Guiqin Wang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 67 Literaturangaben.
Abstract (Originalsprache)
Test-Time Adaptation (TTA) has recently emerged as a promising research direction, enabling vision-language models (VLMs) to adapt to unlabeled test data in zero-shot settings. Among TTA approaches, test-time prompt tuning has shown great potential for enhancing the practical applicability of VLMs. However, existing methods typically either focus on adapting a single modality or apply uniform optimization to both modalities, without explicitly defining modality-specific optimization objectives. Such a one-size-fits-all strategy often results in suboptimal performance under test-time conditions. To address this limitation, we propose Dual-modality Heterogeneous Prompt Tuning (DHPT), a novel framework designed to simultaneously capture fine-grained textual semantics and alleviate domain shift noise in the visual modality. Specifically, we leverage a large language model to provide textual cognition guidance for the text encoder, while on the vision side, we develop a lightweight calibration module that adaptively mitigates domain shift noise across different scales. Furthermore, we introduce a cluster-tight optimization objective that enhances the stability and generalizability of prompt tuning under distribution shifts. Extensive experiments conducted on 11 benchmark datasets demonstrate that DHPT consistently and significantly outperforms existing TTA methods for VLMs.
Zitieren
APA
Wang, G., Zhao, P., Wang, X., Guo, H., Qi, N., Yang, S., & Guo, Q. (2026). DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models. IEEE Transactions on Circuits and Systems for Video Technology, 36(6), 8590-8601. https://doi.org/10.1109/tcsvt.2026.3657756
Vancouver
Wang G, Zhao P, Wang X, Guo H, Qi N, Yang S, et al. DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(6):8590-8601. doi:10.1109/tcsvt.2026.3657756
BibTeX
@article{Wang2026,
author = {Wang, Guiqin and Zhao, Peng and Wang, Xiang and Guo, Haoran and Qi, Nan and Yang, Shusen and Guo, Qinghai},
title = {DHPT: Dual-Modality Heterogeneous Prompt Tuning for Online Test-Time Adaption in Vision-Language Models},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {6},
pages = {8590--8601},
doi = {10.1109/tcsvt.2026.3657756}
}
EUVIMED