EUVIMEDEUVIMED entdecken
Crossref

MagicTime: Time-Lapse Video Generation Models as Metamorphic Simulators

Shenghai Yuan, Jinfa Huang, Yujun Shi, Yongqi Xu, Ruijie Zhu, Bin Lin, Xinhua Cheng, Li Yuan, Jiebo Luo

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-09 · 47 (9) · 7340-7351

DOI: 10.1109/tpami.2025.3558507 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-09. Verfasst von Shenghai Yuan und 8 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 83 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Open research fund of Pengcheng Laboratory.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advances in text-to-video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions. A largely overlooked problem in T2V is that existing models have not adequately encoded physical knowledge of the real world, thus generated videos tend to have limited motion and poor variations. In this paper, we propose MagicTime, a metamorphic time-lapse video generation model, which learns real-world physics knowledge from time-lapse videos and implements metamorphic generation. First, we design a simple yet effective two-stage Magic Adaptive Strategy, encode more physical knowledge from metamorphic videos, and transform pre-trained T2V models to generate metamorphic videos. Second, we introduce a Dynamic Frames Extraction strategy to adapt to metamorphic time-lapse videos, which have a wider variation range and cover dramatic object metamorphic processes, thus embodying more physical knowledge than general videos. Finally, we introduce a Magic Text-Encoder to improve the understanding of metamorphic video prompts. Furthermore, we create a time-lapse video-text dataset called ChronoMagic, specifically curated to unlock the metamorphic video generation ability. Extensive experiments demonstrate the superiority and effectiveness of MagicTime for generating high-quality and dynamic metamorphic videos, suggesting time-lapse video generation is a promising path toward building metamorphic simulators of the physical world.

Förderung

  • National Natural Science Foundation of China
  • Open research fund of Pengcheng Laboratory

Zitieren

APA

Yuan, S., Huang, J., Shi, Y., Xu, Y., Zhu, R., Lin, B., Cheng, X., Yuan, L., & Luo, J. (2025). MagicTime: Time-Lapse Video Generation Models as Metamorphic Simulators. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(9), 7340-7351. https://doi.org/10.1109/tpami.2025.3558507

Vancouver

Yuan S, Huang J, Shi Y, Xu Y, Zhu R, Lin B, et al. MagicTime: Time-Lapse Video Generation Models as Metamorphic Simulators. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(9):7340-7351. doi:10.1109/tpami.2025.3558507

BibTeX
@article{Yuan2025,
  author = {Yuan, Shenghai and Huang, Jinfa and Shi, Yujun and Xu, Yongqi and Zhu, Ruijie and Lin, Bin and Cheng, Xinhua and Yuan, Li and Luo, Jiebo},
  title = {MagicTime: Time-Lapse Video Generation Models as Metamorphic Simulators},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {9},
  pages = {7340--7351},
  doi = {10.1109/tpami.2025.3558507}
}
Kostenlos starten →