Pre-Trained Video Generative Models as World Simulators
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (6) · 4645-4653
DOI: 10.1609/aaai.v40i6.42465 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Haoran He und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref).
Abstract (Originalsprache)
Video generative models pre-trained on large-scale internet datasets have achieved remarkable success, excelling at producing realistic synthetic videos. However, they often generate clips based on static prompts (e.g., text or images), limiting their ability to model interactive and dynamic scenarios. In this paper, we propose Dynamic World Simulation (DWS), a novel approach to transform pre-trained video generative models into controllable world simulators capable of executing specified action trajectories. To achieve precise alignment between conditioned actions and generated visual changes, we introduce a lightweight, universal action-conditioned module that seamlessly integrates into any existing model. Instead of focusing on complex visual details, we demonstrate that consistent dynamic transition modeling is the key to building powerful world simulators. Building upon this insight, we further introduce a motion-reinforced loss that enhances action controllability by compelling the model to capture dynamic changes more effectively. Experiments demonstrate that DWS can be versatilely applied to both diffusion and autoregressive transformer models, achieving significant improvements in generating action-controllable, dynamically consistent videos across games and robotics domains. Moreover, to facilitate the applications of the learned world simulator in downstream tasks such as model-based reinforcement learning, we propose prioritized imagination to improve sample efficiency, demonstrating competitive performance compared with state-of-the-art methods.
Zitieren
APA
He, H., Zhang, Y., Lin, L., Xu, Z., & Pan, L. (2026). Pre-Trained Video Generative Models as World Simulators. Proceedings of the AAAI Conference on Artificial Intelligence, 40(6), 4645-4653. https://doi.org/10.1609/aaai.v40i6.42465
Vancouver
He H, Zhang Y, Lin L, Xu Z, Pan L. Pre-Trained Video Generative Models as World Simulators. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(6):4645-4653. doi:10.1609/aaai.v40i6.42465
BibTeX
@article{He2026,
author = {He, Haoran and Zhang, Yang and Lin, Liang and Xu, Zhongwen and Pan, Ling},
title = {Pre-Trained Video Generative Models as World Simulators},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {6},
pages = {4645--4653},
doi = {10.1609/aaai.v40i6.42465}
}
EUVIMED