VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-03 · 48 (3) · 3268-3285
DOI: 10.1109/tpami.2025.3633890 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-03. Verfasst von Ziqi Huang und 16 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 175 Literaturangaben. Gefördert von: Ministry of Education - Singapore, NTU NAP, Industry Alignment Fund - Industry Collaboration Projects (IAF-ICP) Funding Initiative.
Abstract (Originalsprache)
Video generation has witnessed significant advancements, yet evaluating these models remains a challenge. A comprehensive evaluation benchmark for video generation is indispensable for two reasons: 1) Existing metrics do not fully align with human perceptions; 2) An ideal evaluation system should provide insights to inform future developments of video generation. To this end, we present VBench++, a comprehensive benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions, each with tailored prompts and evaluation methods. VBench++ has several appealing properties: 1) Comprehensive Dimensions: VBench++ comprises 16 dimensions in text-to-video generation (e.g., subject identity inconsistency, motion smoothness, temporal flickering, and spatial relationship, etc). The evaluation metrics with fine-grained levels reveal individual models' strengths and weaknesses. 2) Human Alignment: We also provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception, for each evaluation dimension respectively. 3) Valuable Insights: We look into current models' ability across various evaluation dimensions, and various content types. We also investigate the gaps between video and image generation models. 4) Versatile Benchmarking: VBench++ is designed to evaluate a wide range of video generation tasks, including text-to-video and image-to-video. We introduce a high-quality Image Suite with an adaptive aspect ratio to enable fair evaluations across different image-to-video generation settings. Beyond assessing technical quality, VBench++ evaluates the trustworthiness of video generative models, providing a more holistic view of model performance. 5) Full Open-Sourcing: We fully open-source VBench++, including all prompts, the Image Suite, evaluation methods, generated videos, and human preference annotations.
Förderung
- Ministry of Education - Singapore
- NTU NAP
- Industry Alignment Fund - Industry Collaboration Projects (IAF-ICP) Funding Initiative
- industry partne
- National Key R&D Program of China
- National Natural Science Foundation of China
- Science and Technology Commission of Shanghai Municipality
- Shanghai Artificial lntelligence Laboratory
Zitieren
APA
Huang, Z., Zhang, F., Xu, X., He, Y., Yu, J., Dong, Z., Ma, Q., Chanpaisit, N., Si, C., Jiang, Y., Wang, Y., Chen, X., Chen, Y. C., Wang, L., Lin, D., Qiao, Y., & Liu, Z. (2026). VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(3), 3268-3285. https://doi.org/10.1109/tpami.2025.3633890
Vancouver
Huang Z, Zhang F, Xu X, He Y, Yu J, Dong Z, et al. VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(3):3268-3285. doi:10.1109/tpami.2025.3633890
BibTeX
@article{Huang2026,
author = {Huang, Ziqi and Zhang, Fan and Xu, Xiaojie and He, Yinan and Yu, Jiashuo and Dong, Ziyue and Ma, Qianli and Chanpaisit, Nattapol and Si, Chenyang and Jiang, Yuming and Wang, Yaohui and Chen, Xinyuan and Chen, Ying-Cong and Wang, Limin and Lin, Dahua and Qiao, Yu and Liu, Ziwei},
title = {VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {3},
pages = {3268--3285},
doi = {10.1109/tpami.2025.3633890}
}
EUVIMED