DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (4) · 3733-3741
DOI: 10.1609/aaai.v39i4.32389 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Tianyu Huang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref).
Abstract (Originalsprache)
Dynamic 3D interaction has been attracting a lot of attention recently. However, creating such 4D content remains challenging. One solution is to animate 3D scenes with physics-based simulation, which requires manually assigning precise physical properties to the object or the simulated results would become unnatural. Another solution is to learn the deformation of 3D objects with the distillation of video generative models, which, however, tends to produce 3D videos with small and discontinuous motions due to the inappropriate extraction and application of physics priors. In this work, to combine the strengths and complementing shortcomings of the above two solutions, we propose to learn the physical properties of a material field with video diffusion priors, and then utilize a physics-based Material-Point-Method (MPM) simulator to generate 4D content with realistic motions. In particular, we propose motion distillation sampling to emphasize video motion information during distillation. In addition, to facilitate the optimization, we further propose a KAN-based material field with frame boosting. Experimental results demonstrate that our method enjoys more realistic motions than state-of-the-arts do.
Zitieren
APA
Huang, T., Zhang, H., Zeng, Y., Zhang, Z., Li, H., Zuo, W., & Lau, R. W. H. (2025). DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors. Proceedings of the AAAI Conference on Artificial Intelligence, 39(4), 3733-3741. https://doi.org/10.1609/aaai.v39i4.32389
Vancouver
Huang T, Zhang H, Zeng Y, Zhang Z, Li H, Zuo W, et al. DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(4):3733-3741. doi:10.1609/aaai.v39i4.32389
BibTeX
@article{Huang2025,
author = {Huang, Tianyu and Zhang, Haoze and Zeng, Yihan and Zhang, Zhilu and Li, Hui and Zuo, Wangmeng and Lau, Rynson W. H.},
title = {DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {4},
pages = {3733--3741},
doi = {10.1609/aaai.v39i4.32389}
}
EUVIMED