LiveLight: Real-time Streaming Video Relighting with Interactive Control
ACM Transactions on Graphics · 09.11.2026 · 45 (6) · 1-16
DOI: 10.1145/3842529 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 09.11.2026. Verfasst von Yue Ma und 10 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 72 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
We present LiveLight, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control. Achieving this is non-trivial, as it requires overcoming three critical challenges: effectively injecting dynamic 3D lighting into a diffusion model, maintaining high-fidelity generation under an extremely low NFE (Number of Function Evaluations) budget for real-time speed, and facilitating continuous streaming for interactive control. To address these pain points, we propose three key designs. First, for accurate lighting injection, we propose a lightweight adapter that feeds Multi-Plane Light Irradiance (MPLI) conditions—depth-aware irradiance maps encoding 3D lighting geometry—directly into the diffusion backbone. Second, to prevent rendering quality degradation at low NFEs towards real-time distillation, we introduce a geometry-guided feedback branch. This training-time constraint leverages a frozen geometry estimator to enforce depth- and normal-consistent relighting, ensuring geometrically plausible shading without adding inference overhead. Finally, to enable streaming interaction, we develop a progressive rolling-window strategy that maintains a denoising ladder of latent chunks at varying noise levels. By propagating intermediate states, this strategy guarantees temporal coherence and supports arbitrarily long video relighting with per-frame reference refresh. Extensive experiments on real-world and synthetic benchmarks demonstrate that LiveLight achieves state-of-the-art relighting quality while running at real-time speed, significantly outperforming offline baselines in temporal stability, lighting controllability, and user preference. To foster real-time interactive relighting research, we will publicly release our models, training data, and synthetic data generator on www.dummy.url.
Zitieren
APA
Ma, Y., Wang, J., Wang, Y., Wang, X., Li, Z., Wang, X., Liu, H., Liang, R., Zhang, S., Xue, Y., & Chen, Q. (2026). LiveLight: Real-time Streaming Video Relighting with Interactive Control. ACM Transactions on Graphics, 45(6), 1-16. https://doi.org/10.1145/3842529
Vancouver
Ma Y, Wang J, Wang Y, Wang X, Li Z, Wang X, et al. LiveLight: Real-time Streaming Video Relighting with Interactive Control. ACM Transactions on Graphics. 2026;45(6):1-16. doi:10.1145/3842529
BibTeX
@article{Ma2026,
author = {Ma, Yue and Wang, Jiangming and Wang, Yucheng and Wang, Xilai and Li, Zhiyuan and Wang, Xinyu and Liu, Hongyu and Liang, Ruofan and Zhang, Songchun and Xue, Yuxuan and Chen, Qifeng},
title = {LiveLight: Real-time Streaming Video Relighting with Interactive Control},
journal = {ACM Transactions on Graphics},
year = {2026},
volume = {45},
number = {6},
pages = {1--16},
doi = {10.1145/3842529}
}
EUVIMED