EUVIMEDEUVIMED entdecken
Crossref

Zero-to-Hero: Empowering Video Appearance Transfer with Zero-Shot Initialization and Holistic Restoration

Tongtong Su, Chengyu Wang, Haipeng Liao, Jun Huang, Dongming Lu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (11) · 9153-9161

DOI: 10.1609/aaai.v40i11.37872 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Tongtong Su und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Appearance editing according to user needs is a pivotal task in video editing. Existing text-guided methods often lead to ambiguities regarding user intentions and restrict fine-grained control over editing specific aspects of objects. To overcome these limitations, this paper introduces a novel approach named Zero-to-Hero, which focuses on reference-based video editing by disentangling the editing process into two distinct problems. It achieves this by first editing an anchor frame to satisfy user requirements as a reference image and then consistently propagating its appearance across the other frames in the video. To achieve accurate appearance propagation, in the first stage of Zero-to-Hero, we leverage correspondences within the original frames to guide the attention mechanism, which is more robust than previously proposed optical flow or temporal modules in memory-friendly video generative models, especially when dealing with objects exhibiting large motions. This offers a solid zero-shot initialization that ensures both accuracy and temporal consistency. However, intervention in the attention mechanism results in compounded imaging degradation with unknown blurring and color-missing issues. Following the Zero-Stage, our Hero-Stage holistically learns a conditional generative model for video restoration. To accurately evaluate appearance consistency, we construct a set of videos with multiple appearances using Blender, enabling a fine-grained and deterministic evaluation. Our method outperforms the best-performing baseline with a PSNR improvement of 2.6 dB.

Zitieren

APA

Su, T., Wang, C., Liao, H., Huang, J., & Lu, D. (2026). Zero-to-Hero: Empowering Video Appearance Transfer with Zero-Shot Initialization and Holistic Restoration. Proceedings of the AAAI Conference on Artificial Intelligence, 40(11), 9153-9161. https://doi.org/10.1609/aaai.v40i11.37872

Vancouver

Su T, Wang C, Liao H, Huang J, Lu D. Zero-to-Hero: Empowering Video Appearance Transfer with Zero-Shot Initialization and Holistic Restoration. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(11):9153-9161. doi:10.1609/aaai.v40i11.37872

BibTeX
@article{Su2026,
  author = {Su, Tongtong and Wang, Chengyu and Liao, Haipeng and Huang, Jun and Lu, Dongming},
  title = {Zero-to-Hero: Empowering Video Appearance Transfer with Zero-Shot Initialization and Holistic Restoration},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {11},
  pages = {9153--9161},
  doi = {10.1609/aaai.v40i11.37872}
}
Kostenlos starten →