EUVIMEDEUVIMED entdecken
Crossref

Video Diffusion Models Are Strong Video Inpainter

Minhyeok Lee, Suhwan Cho, Chajin Shin, Jungho Lee, Sunghun Yang, Sangyoun Lee

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (4) · 4526-4533

DOI: 10.1609/aaai.v39i4.32477 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Minhyeok Lee und 5 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Propagation-based video inpainting using optical flow at the pixel or feature level has recently garnered significant attention. However, it has limitations such as the inaccuracy of optical flow prediction and the propagation of noise over time. These issues result in non-uniform noise and time consistency problems throughout the video, which are particularly pronounced when the removed area is large and involves substantial movement. To address these issues, we propose a novel First Frame Filling Video Diffusion Inpainting model (FFF-VDI). We design FFF-VDI inspired by the capabilities of pre-trained image-to-video diffusion models that can transform the first frame image into a highly natural video. To apply this to the video inpainting task, we propagate the noise latent information of future frames to fill the masked areas of the first frame's noise latent code. Next, we fine-tune the pre-trained image-to-video diffusion model to generate the inpainted video. The proposed model addresses the limitations of existing methods that rely on optical flow quality, producing much more natural and temporally consistent videos. This proposed approach is the first to effectively integrate image-to-video diffusion models into video inpainting tasks. Through various comparative experiments, we demonstrate that the proposed model can robustly handle diverse inpainting types with high quality.

Zitieren

APA

Lee, M., Cho, S., Shin, C., Lee, J., Yang, S., & Lee, S. (2025). Video Diffusion Models Are Strong Video Inpainter. Proceedings of the AAAI Conference on Artificial Intelligence, 39(4), 4526-4533. https://doi.org/10.1609/aaai.v39i4.32477

Vancouver

Lee M, Cho S, Shin C, Lee J, Yang S, Lee S. Video Diffusion Models Are Strong Video Inpainter. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(4):4526-4533. doi:10.1609/aaai.v39i4.32477

BibTeX
@article{Lee2025,
  author = {Lee, Minhyeok and Cho, Suhwan and Shin, Chajin and Lee, Jungho and Yang, Sunghun and Lee, Sangyoun},
  title = {Video Diffusion Models Are Strong Video Inpainter},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {4},
  pages = {4526--4533},
  doi = {10.1609/aaai.v39i4.32477}
}
Kostenlos starten →