DiFaReli++: Diffusion Face Relighting With Consistent Cast Shadows
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-05 · 48 (5) · 5068-5082
DOI: 10.1109/tpami.2025.3648667 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-05. Verfasst von Puntawat Ponglertnapakorn und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 74 Literaturangaben. Gefördert von: PTT Public Company Limited, SCB Public Company Limited.
Abstract (Originalsprache)
We introduce a novel approach to single-view face relighting in the wild, addressing challenges such as global illumination and cast shadows. A common scheme in recent methods involves intrinsicallydecomposing an input image into 3D shape, albedo, and lighting, then recomposing it with the target lighting. However, estimating these components is error-prone and requires many training examples with ground-truth lighting to generalize well. Our work bypasses the need for accurate intrinsic estimation and can be trained solely on 2D images without any light stage data, relit pairs, multi-view images, or lighting ground truth. Our key idea is to leverage a conditional diffusion implicit model (DDIM) for decoding a disentangled light encoding along with other encodings related to 3D shape and facial identity inferred from off-the-shelf estimators. We propose a novel conditioning technique that simplifies modeling the complex interaction between light and geometry. It uses a rendered shading reference along with a shadow map, inferred using a simple and effective technique, to spatially modulate the DDIM. Moreover, we propose a single-shot relighting framework that requires just one network pass, given pre-processed data, and even outperforms the teacher model across all metrics. Our method realistically relights in-the-wild images with temporally consistent cast shadows under varying lighting conditions. We achieve state-of-the-art performance on the standard benchmark Multi-PIE and rank highest in user studies. Please visit our page: https://diffusion-face-relighting-pp.github.io.
Förderung
- PTT Public Company Limited
- SCB Public Company Limited
Zitieren
APA
Ponglertnapakorn, P., Tritrong, N., & Suwajanakorn, S. (2026). DiFaReli++: Diffusion Face Relighting With Consistent Cast Shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(5), 5068-5082. https://doi.org/10.1109/tpami.2025.3648667
Vancouver
Ponglertnapakorn P, Tritrong N, Suwajanakorn S. DiFaReli++: Diffusion Face Relighting With Consistent Cast Shadows. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(5):5068-5082. doi:10.1109/tpami.2025.3648667
BibTeX
@article{Ponglertnapakorn2026,
author = {Ponglertnapakorn, Puntawat and Tritrong, Nontawat and Suwajanakorn, Supasorn},
title = {DiFaReli++: Diffusion Face Relighting With Consistent Cast Shadows},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {5},
pages = {5068--5082},
doi = {10.1109/tpami.2025.3648667}
}
EUVIMED