StoryVideoQA: Scaling Deep Video Understanding with a Large-Scale, Multi-Genre and Auto-Generated Dataset
International Journal of Computer Vision · 2026-06 · 134 (6)
DOI: 10.1007/s11263-026-02898-w ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 04.06.2026. Verfasst von Zhengqian Wu und 8 weiteren Autor:innen. Die Arbeit nennt 112 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Ministry of Education Industry-University Cooperative Education Project, Key Science and Technology Research Project of Xinjiang Production and Construction Corps.
Abstract (Originalsprache)
Video question answering (VideoQA) aims to answer questions about given videos. While existing approaches excel on factoid VideoQA, they struggle with deep video understanding (DVU), which requires the comprehension of complex storylines. This challenge arises from the inherent long-range video content, multi-faceted question types, and instance-level story elements, all of which constrain the scale and diversity of manually constructed DVU datasets. These difficulties constrain the scale and diversity of manually-constructed DVU dataset. To address these, we previously introduced StoryMind to automatically construct DVU datasets with balanced fine-grained topics. Though it can generate high-quality question-answer pairs (QAs) for TV series, it suffers significant performance degradation when handling longer and more complex movies. In this paper, we further design StoryMindv2, an enhanced multi-agent collaboration framework to generate high-quality DVU datasets for both TV series and movies. By integrating a novel supervisor-guided generation mechanism and a refined multi-reviewer voting strategy, the framework is utilized to construct StoryVideoQA, the largest DVU dataset to date, featuring over 363K QAs on 393.2 hours diverse story videos including TV series (avg. 1,635 seconds) and movies (avg. 7,878 seconds). Comprehensive evaluations of 20 state-of-the-art VideoQA methods on this large-scale benchmark reveal that they cannot fully maintain long-range character associations or construct a coherent understanding of complex storylines. To bridge this gap, we propose PlotTree, a novel video understanding agent, re-organizing long-range video content into a hierarchical plot structure, enabling efficient storyline reasoning on StoryVideoQA. Project page: https://github.com/nercms-mmap/StoryVideoQA/.
Förderung
- National Natural Science Foundation of China
- Ministry of Education Industry-University Cooperative Education Project
- Key Science and Technology Research Project of Xinjiang Production and Construction Corps
- Hubei Provincial Science and Technology Plan Project
Zitieren
APA
Wu, Z., Liu, Z., Chen, A., Zhang, J., Li, R., Ge, H., Wang, Z., Xiao, C., & Liang, C. (2026). StoryVideoQA: Scaling Deep Video Understanding with a Large-Scale, Multi-Genre and Auto-Generated Dataset. International Journal of Computer Vision, 134(6). https://doi.org/10.1007/s11263-026-02898-w
Vancouver
Wu Z, Liu Z, Chen A, Zhang J, Li R, Ge H, et al. StoryVideoQA: Scaling Deep Video Understanding with a Large-Scale, Multi-Genre and Auto-Generated Dataset. International Journal of Computer Vision. 2026;134(6). doi:10.1007/s11263-026-02898-w
BibTeX
@article{Wu2026,
author = {Wu, Zhengqian and Liu, Zhixian and Chen, Aodong and Zhang, Jingyang and Li, Ruizhe and Ge, Hanlin and Wang, Zhongyuan and Xiao, Chunxia and Liang, Chao},
title = {StoryVideoQA: Scaling Deep Video Understanding with a Large-Scale, Multi-Genre and Auto-Generated Dataset},
journal = {International Journal of Computer Vision},
year = {2026},
volume = {134},
number = {6},
doi = {10.1007/s11263-026-02898-w}
}
EUVIMED