EUVIMEDEUVIMED entdecken
Crossref

ProtoDCS: Toward Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models

Wei Luo, Yangfan Ou, Jin Deng, Zeshuai Deng, Xiquan Yan, Zhiquan Wen, Mingkui Tan

IEEE Transactions on Circuits and Systems for Video Technology · 2026-11 · 36 (11) · 16963-16977

DOI: 10.1109/tcsvt.2026.3717446 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-11. Verfasst von Wei Luo und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Joint Funds of the National Natural Science Foundation of China, Young Scholar Project of Pazhou Laboratory.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large-scale Vision-Language Models (VLMs) exhibit strong zero-shot recognition, yet their real-world deployment is challenged by distribution shifts. While Test-Time Adaptation (TTA) can mitigate this, existing VLM-based TTA methods operate under a closed-set assumption, failing in open-set scenarios where test streams contain both covariate-shifted in-distribution (csID) and out-of-distribution (csOOD) data. This leads to a critical difficulty: the model must discriminate unknown csOOD samples to avoid interference while simultaneously adapting to known csID classes for accuracy. Current open-set TTA (OSTTA) methods rely on hard thresholds for separation and entropy minimization for adaptation. These strategies are brittle, often misclassifying ambiguous csOOD samples and inducing overconfident predictions, and their parameter-update mechanism is computationally prohibitive for VLMs. To address these limitations, we propose Prototype-based Double-Check Separation (ProtoDCS), a robust framework for OSTTA that effectively separates csID and csOOD samples, enabling safe and efficient adaptation of VLMs to csID data. Our main contributions are: (1) a novel double-check separation mechanism employing probabilistic Gaussian Mixture Model (GMM) verification to mitigate the brittleness of rigid thresholding; and (2) an evidence-driven adaptation strategy utilizing uncertainty-aware loss and efficient prototype-level updates, mitigating overconfidence and reducing computational overhead. Extensive experiments on CIFAR-10/100-C and Tiny-ImageNet-C demonstrate that ProtoDCS achieves state-of-the-art performance, significantly boosting both known-class accuracy and OOD detection metrics. Code will be available at https://github.com/O-YangF/ProtoDCS

Förderung

  • National Natural Science Foundation of China
  • Joint Funds of the National Natural Science Foundation of China
  • Young Scholar Project of Pazhou Laboratory

Zitieren

APA

Luo, W., Ou, Y., Deng, J., Deng, Z., Yan, X., Wen, Z., & Tan, M. (2026). ProtoDCS: Toward Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models. IEEE Transactions on Circuits and Systems for Video Technology, 36(11), 16963-16977. https://doi.org/10.1109/tcsvt.2026.3717446

Vancouver

Luo W, Ou Y, Deng J, Deng Z, Yan X, Wen Z, et al. ProtoDCS: Toward Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(11):16963-16977. doi:10.1109/tcsvt.2026.3717446

BibTeX
@article{Luo2026,
  author = {Luo, Wei and Ou, Yangfan and Deng, Jin and Deng, Zeshuai and Yan, Xiquan and Wen, Zhiquan and Tan, Mingkui},
  title = {ProtoDCS: Toward Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2026},
  volume = {36},
  number = {11},
  pages = {16963--16977},
  doi = {10.1109/tcsvt.2026.3717446}
}
Kostenlos starten →