Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing
IEEE Transactions on Circuits and Systems for Video Technology · 2025-06 · 35 (6) · 6034-6046
DOI: 10.1109/tcsvt.2025.3531390 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-06. Verfasst von Ling Yang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 71 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key Research and Development Program of China.
Abstract (Originalsprache)
Diffusion-based generative models have exhibited considerable success in conditional video synthesis and editing. Nevertheless, prevailing video diffusion models primarily rely on conditioning with specific input modalities, predominantly text, restricting their adaptability to alternative modalities without necessitating retraining of modality-specific components. In this work, we present EnergyViD, a universal spatio-temporal Energy-guided Video Diffusion model designed for zero-shot video synthesis and editing across diverse conditions. Specifically, we leverage off-the-shelf pre-trained networks to construct generic energy functions, guiding the generation process under specific conditions without the need for retraining. To precisely capture temporal dynamics related to motion conditions (e.g., pose sequences), we introduce a novel kernel Maximum Mean Discrepancy (MMD)-based energy function, which minimizes the global distribution discrepancy between the conditioning input and the generated video. Our extensive qualitative and quantitative experiments demonstrate that our algorithm consistently produces high-quality results across a wide range of motion and non-motion conditions, including text, face ID, style, poses, depths, sketches, canny edges, and segmentation maps, in the context of zero-shot video synthesis and editing. We will release source code upon acceptance of the paper.
Förderung
- National Natural Science Foundation of China
- National Key Research and Development Program of China
Zitieren
APA
Yang, L., Zhao, Y., Yu, Z., Zeng, B., Xu, M., Hong, S., & Cui, B. (2025). Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing. IEEE Transactions on Circuits and Systems for Video Technology, 35(6), 6034-6046. https://doi.org/10.1109/tcsvt.2025.3531390
Vancouver
Yang L, Zhao Y, Yu Z, Zeng B, Xu M, Hong S, et al. Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(6):6034-6046. doi:10.1109/tcsvt.2025.3531390
BibTeX
@article{Yang2025,
author = {Yang, Ling and Zhao, Yikai and Yu, Zhaochen and Zeng, Bohan and Xu, Minkai and Hong, Shenda and Cui, Bin},
title = {Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2025},
volume = {35},
number = {6},
pages = {6034--6046},
doi = {10.1109/tcsvt.2025.3531390}
}
EUVIMED