EUVIMEDEUVIMED entdecken
Crossref

Revisiting Classifier: Transferring Vision-Language Models for Video Recognition

Wenhao Wu, Zhun Sun, Wanli Ouyang

Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (3) · 2847-2855

DOI: 10.1609/aaai.v37i3.25386 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Wenhao Wu und 2 weiteren Autor:innen. Die Arbeit wurde bisher 96-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transferring knowledge from task-agnostic pre-trained deep models for downstream tasks is an important topic in computer vision research. Along with the growth of computational capacity, we now have open-source vision-language pre-trained models in large scales of the model architecture and amount of data. In this study, we focus on transferring knowledge for video classification tasks. Conventional methods randomly initialize the linear classifier head for vision classification, but they leave the usage of the text encoder for downstream visual recognition tasks undiscovered. In this paper, we revise the role of the linear classifier and replace the classifier with the different knowledge from pre-trained model. We utilize the well-pretrained language model to generate good semantic target for efficient transferring learning. The empirical study shows that our method improves both the performance and the training speed of video classification, with a negligible change in the model. Our simple yet effective tuning paradigm achieves state-of-the-art performance and efficient training on various video recognition scenarios, i.e., zero-shot, few-shot, general recognition. In particular, our paradigm achieves the state-of-the-art accuracy of 87.8% on Kinetics-400, and also surpasses previous methods by 20~50% absolute top-1 accuracy under zero-shot, few-shot settings on five video datasets. Code and models are available at https://github.com/whwu95/Text4Vis.

Zitieren

APA

Wu, W., Sun, Z., & Ouyang, W. (2023). Revisiting Classifier: Transferring Vision-Language Models for Video Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 37(3), 2847-2855. https://doi.org/10.1609/aaai.v37i3.25386

Vancouver

Wu W, Sun Z, Ouyang W. Revisiting Classifier: Transferring Vision-Language Models for Video Recognition. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(3):2847-2855. doi:10.1609/aaai.v37i3.25386

BibTeX
@article{Wu2023,
  author = {Wu, Wenhao and Sun, Zhun and Ouyang, Wanli},
  title = {Revisiting Classifier: Transferring Vision-Language Models for Video Recognition},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2023},
  volume = {37},
  number = {3},
  pages = {2847--2855},
  doi = {10.1609/aaai.v37i3.25386}
}
Kostenlos starten →