EUVIMEDEUVIMED entdecken
Crossref

CLIPVQA: Video Quality Assessment via CLIP

Fengchuang Xing, Mingjie Li, Yuan-Gen Wang, Guopu Zhu, Xiaochun Cao

IEEE Transactions on Broadcasting · 2025-03 · 71 (1) · 291-306

DOI: 10.1109/tbc.2024.3511927 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Broadcasting, 2025-03. Verfasst von Fengchuang Xing und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 79 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Guangdong Provincial Natural Science Foundation, Guangzhou Municipal Science and Technology Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In learning vision-language representations from Web-scale data, the contrastive language-image pre-training (CLIP) mechanism has demonstrated a remarkable performance in many vision tasks. However, its application to the widely studied video quality assessment (VQA) task is still an open issue. In this paper, we propose an efficient and effective CLIP-based Transformer method for the VQA problem (CLIPVQA). Specifically, we first design an effective video frame perception paradigm with the goal of extracting the rich spatiotemporal quality and content information among video frames. Then, the spatiotemporal quality features are adequately integrated together using a self-attention mechanism to yield video-level quality representation. To utilize the quality language descriptions of videos for supervision, we develop a CLIP-based encoder for language embedding, which is then fully aggregated with the generated content information via a cross-attention module for producing video-language representation. Finally, the video-level quality and video-language representations are fused together for final video quality prediction, where a vectorized regression loss is employed for efficient end-to-end optimization. Comprehensive experiments are conducted on eight in-the-wild video datasets with diverse resolutions to evaluate the performance of CLIPVQA. The experimental results show that the proposed CLIPVQA achieves new state-of-the-art VQA performance and up to 37% better generalizability than existing benchmark VQA methods. A series of ablation studies are also performed to validate the effectiveness of each module in CLIPVQA.

Förderung

  • National Natural Science Foundation of China
  • Guangdong Provincial Natural Science Foundation
  • Guangzhou Municipal Science and Technology Project
  • Fundamental Research Funds for the Central Universities
  • Science Foundation of Guangzhou University
  • Guangdong Scientific Research Platform and Projects for Higher Educational Institution

Zitieren

APA

Xing, F., Li, M., Wang, Y., Zhu, G., & Cao, X. (2025). CLIPVQA: Video Quality Assessment via CLIP. IEEE Transactions on Broadcasting, 71(1), 291-306. https://doi.org/10.1109/tbc.2024.3511927

Vancouver

Xing F, Li M, Wang Y, Zhu G, Cao X. CLIPVQA: Video Quality Assessment via CLIP. IEEE Transactions on Broadcasting. 2025;71(1):291-306. doi:10.1109/tbc.2024.3511927

BibTeX
@article{Xing2025,
  author = {Xing, Fengchuang and Li, Mingjie and Wang, Yuan-Gen and Zhu, Guopu and Cao, Xiaochun},
  title = {CLIPVQA: Video Quality Assessment via CLIP},
  journal = {IEEE Transactions on Broadcasting},
  year = {2025},
  volume = {71},
  number = {1},
  pages = {291--306},
  doi = {10.1109/tbc.2024.3511927}
}
Kostenlos starten →