EUVIMEDEUVIMED entdecken
Crossref

Text2Relight: Creative Portrait Relighting with Text Guidance

Junuk Cha, Mengwei Ren, Krishna Kumar Singh, He Zhang, Yannick Hold-Geoffroy, Seunghyun Yoon, HyunJoon Jung, Jae Shin Yoon, Seungryul Baek

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (2) · 1980-1988

DOI: 10.1609/aaai.v39i2.32194 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Junuk Cha und 8 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present a lighting-aware image editing pipeline that, given a portrait image and a text prompt, performs single image relighting. Our model modifies the lighting and color of both the foreground and background to align with the provided text description. The unbounded nature in creativeness of a text allows us to describe the lighting of a scene with any sensory features including temperature, emotion, smell, time, and so on. However, the modeling of such mapping between the unbounded text and lighting is extremely challenging due to the lack of dataset where there exists no scalable data that provides large pairs of text and relighting, and therefore, current text-driven image editing models does not generalize to lighting-specific use cases. We overcome this problem by introducing a novel data synthesis pipeline: First, diverse and creative text prompts that describe the scenes with various lighting are automatically generated under a crafted hierarchy using a large language model (e.g., ChatGPT). A text-guided image generation model creates a lighting image that best matches the text. As a condition of the lighting images, we perform image-based relighting for both foreground and background using a single portrait image or a set of OLAT (One-Light-at-A-Time) images captured from lightstage system. Particularly for the background relighting, we represent the lighting image as a set of point lights and transfer them to other background images. A generative diffusion model learns the synthesized large-scale data with auxiliary task augmentation (e.g., portrait delighting and light positioning) to correlate the latent text and lighting distribution for text-guided portrait relighting. In our experiment, we demonstrate that our model outperforms existing text-guided image generation models, showing high-quality portrait relighting results with a strong generalization to unconstrained scenes.

Zitieren

APA

Cha, J., Ren, M., Singh, K. K., Zhang, H., Hold-Geoffroy, Y., Yoon, S., Jung, H., Yoon, J. S., & Baek, S. (2025). Text2Relight: Creative Portrait Relighting with Text Guidance. Proceedings of the AAAI Conference on Artificial Intelligence, 39(2), 1980-1988. https://doi.org/10.1609/aaai.v39i2.32194

Vancouver

Cha J, Ren M, Singh KK, Zhang H, Hold-Geoffroy Y, Yoon S, et al. Text2Relight: Creative Portrait Relighting with Text Guidance. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(2):1980-1988. doi:10.1609/aaai.v39i2.32194

BibTeX
@article{Cha2025,
  author = {Cha, Junuk and Ren, Mengwei and Singh, Krishna Kumar and Zhang, He and Hold-Geoffroy, Yannick and Yoon, Seunghyun and Jung, HyunJoon and Yoon, Jae Shin and Baek, Seungryul},
  title = {Text2Relight: Creative Portrait Relighting with Text Guidance},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {2},
  pages = {1980--1988},
  doi = {10.1609/aaai.v39i2.32194}
}
Kostenlos starten →