EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

LiveLight: Real-time Streaming Video Relighting with Interactive Control

Yue Ma, Jiangming Wang, Yucheng Wang, Xilai Wang, Zhiyuan Li, Xinyu Wang, Hongyu Liu, Ruofan Liang, Songchun Zhang, Yuxuan Xue, Qifeng Chen

ACM Transactions on Graphics · 09.11.2026 · 45 (6) · 1-16

DOI: 10.1145/3842529 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 09.11.2026. Verfasst von Yue Ma und 10 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 72 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present LiveLight, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control. Achieving this is non-trivial, as it requires overcoming three critical challenges: effectively injecting dynamic 3D lighting into a diffusion model, maintaining high-fidelity generation under an extremely low NFE (Number of Function Evaluations) budget for real-time speed, and facilitating continuous streaming for interactive control. To address these pain points, we propose three key designs. First, for accurate lighting injection, we propose a lightweight adapter that feeds Multi-Plane Light Irradiance (MPLI) conditions—depth-aware irradiance maps encoding 3D lighting geometry—directly into the diffusion backbone. Second, to prevent rendering quality degradation at low NFEs towards real-time distillation, we introduce a geometry-guided feedback branch. This training-time constraint leverages a frozen geometry estimator to enforce depth- and normal-consistent relighting, ensuring geometrically plausible shading without adding inference overhead. Finally, to enable streaming interaction, we develop a progressive rolling-window strategy that maintains a denoising ladder of latent chunks at varying noise levels. By propagating intermediate states, this strategy guarantees temporal coherence and supports arbitrarily long video relighting with per-frame reference refresh. Extensive experiments on real-world and synthetic benchmarks demonstrate that LiveLight achieves state-of-the-art relighting quality while running at real-time speed, significantly outperforming offline baselines in temporal stability, lighting controllability, and user preference. To foster real-time interactive relighting research, we will publicly release our models, training data, and synthetic data generator on www.dummy.url.

Zitieren

APA

Ma, Y., Wang, J., Wang, Y., Wang, X., Li, Z., Wang, X., Liu, H., Liang, R., Zhang, S., Xue, Y., & Chen, Q. (2026). LiveLight: Real-time Streaming Video Relighting with Interactive Control. ACM Transactions on Graphics, 45(6), 1-16. https://doi.org/10.1145/3842529

Vancouver

Ma Y, Wang J, Wang Y, Wang X, Li Z, Wang X, et al. LiveLight: Real-time Streaming Video Relighting with Interactive Control. ACM Transactions on Graphics. 2026;45(6):1-16. doi:10.1145/3842529

BibTeX
@article{Ma2026,
  author = {Ma, Yue and Wang, Jiangming and Wang, Yucheng and Wang, Xilai and Li, Zhiyuan and Wang, Xinyu and Liu, Hongyu and Liang, Ruofan and Zhang, Songchun and Xue, Yuxuan and Chen, Qifeng},
  title = {LiveLight: Real-time Streaming Video Relighting with Interactive Control},
  journal = {ACM Transactions on Graphics},
  year = {2026},
  volume = {45},
  number = {6},
  pages = {1--16},
  doi = {10.1145/3842529}
}
Kostenlos starten →