EUVIMEDEUVIMED entdecken
Crossref

A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis

Y. Poirier‐Ginter, A. Gauthier, J. Phillip, J.‐F. Lalonde, G. Drettakis

Computer Graphics Forum · 2024-07 · 43 (4)

DOI: 10.1111/cgf.15147 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computer Graphics Forum, online erschienen am 24.07.2024. Verfasst von Y. Poirier‐Ginter und 4 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 82 Literaturangaben. Gefördert von: European Resuscitation Council, Natural Sciences and Engineering Research Council of Canada, Alliance de recherche numérique du Canada.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Relighting radiance fields is severely underconstrained for multi‐view data, which is most often captured under a single illumination condition; It is especially hard for full scenes containing multiple objects. We introduce a method to create relightable radiance fields using such single‐illumination data by exploiting priors extracted from 2D image diffusion models. We first fine‐tune a 2D diffusion model on a multi‐illumination dataset conditioned by light direction, allowing us to augment a single‐illumination capture into a realistic – but possibly inconsistent – multi‐illumination dataset from directly defined light directions. We use this augmented data to create a relightable radiance field represented by 3D Gaussian splats. To allow direct control of light direction for low‐frequency lighting, we represent appearance with a multi‐layer perceptron parameterized on light direction. To enforce multi‐view consistency and overcome inaccuracies we optimize a per‐image auxiliary feature vector. We show results on synthetic and real multi‐view data under single illumination, demonstrating that our method successfully exploits 2D diffusion model priors to allow realistic 3D relighting for complete scenes.

Förderung

  • European Resuscitation Council
  • Natural Sciences and Engineering Research Council of Canada
  • Alliance de recherche numérique du Canada

Zitieren

APA

Poirier‐Ginter, Y., Gauthier, A., Phillip, J., Lalonde, J., & Drettakis, G. (2024). A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis. Computer Graphics Forum, 43(4). https://doi.org/10.1111/cgf.15147

Vancouver

Poirier‐Ginter Y, Gauthier A, Phillip J, Lalonde J, Drettakis G. A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis. Computer Graphics Forum. 2024;43(4). doi:10.1111/cgf.15147

BibTeX
@article{PoirierGinter2024,
  author = {Poirier‐Ginter, Y. and Gauthier, A. and Phillip, J. and Lalonde, J.‐F. and Drettakis, G.},
  title = {A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis},
  journal = {Computer Graphics Forum},
  year = {2024},
  volume = {43},
  number = {4},
  doi = {10.1111/cgf.15147}
}
Kostenlos starten →