EUVIMEDEUVIMED entdecken
Crossref

Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing

Ling Yang, Yikai Zhao, Zhaochen Yu, Bohan Zeng, Minkai Xu, Shenda Hong, Bin Cui

IEEE Transactions on Circuits and Systems for Video Technology · 2025-06 · 35 (6) · 6034-6046

DOI: 10.1109/tcsvt.2025.3531390 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-06. Verfasst von Ling Yang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 71 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Diffusion-based generative models have exhibited considerable success in conditional video synthesis and editing. Nevertheless, prevailing video diffusion models primarily rely on conditioning with specific input modalities, predominantly text, restricting their adaptability to alternative modalities without necessitating retraining of modality-specific components. In this work, we present EnergyViD, a universal spatio-temporal Energy-guided Video Diffusion model designed for zero-shot video synthesis and editing across diverse conditions. Specifically, we leverage off-the-shelf pre-trained networks to construct generic energy functions, guiding the generation process under specific conditions without the need for retraining. To precisely capture temporal dynamics related to motion conditions (e.g., pose sequences), we introduce a novel kernel Maximum Mean Discrepancy (MMD)-based energy function, which minimizes the global distribution discrepancy between the conditioning input and the generated video. Our extensive qualitative and quantitative experiments demonstrate that our algorithm consistently produces high-quality results across a wide range of motion and non-motion conditions, including text, face ID, style, poses, depths, sketches, canny edges, and segmentation maps, in the context of zero-shot video synthesis and editing. We will release source code upon acceptance of the paper.

Förderung

  • National Natural Science Foundation of China
  • National Key Research and Development Program of China

Zitieren

APA

Yang, L., Zhao, Y., Yu, Z., Zeng, B., Xu, M., Hong, S., & Cui, B. (2025). Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing. IEEE Transactions on Circuits and Systems for Video Technology, 35(6), 6034-6046. https://doi.org/10.1109/tcsvt.2025.3531390

Vancouver

Yang L, Zhao Y, Yu Z, Zeng B, Xu M, Hong S, et al. Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(6):6034-6046. doi:10.1109/tcsvt.2025.3531390

BibTeX
@article{Yang2025,
  author = {Yang, Ling and Zhao, Yikai and Yu, Zhaochen and Zeng, Bohan and Xu, Minkai and Hong, Shenda and Cui, Bin},
  title = {Spatio-Temporal Energy-Guided Diffusion Model for Zero-Shot Video Synthesis and Editing},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {6},
  pages = {6034--6046},
  doi = {10.1109/tcsvt.2025.3531390}
}
Kostenlos starten →