EUVIMEDEUVIMED entdecken
Crossref

SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-Form Layout-to-Image Generation

Chengyou Jia, Minnan Luo, Zhuohang Dang, Guang Dai, Xiaojun Chang, Mengmeng Wang, Jingdong Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (3) · 2480-2488

DOI: 10.1609/aaai.v38i3.28024 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Chengyou Jia und 6 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite significant progress in Text-to-Image (T2I) generative models, even lengthy and complex text descriptions still struggle to convey detailed controls. In contrast, Layout-to-Image (L2I) generation, aiming to generate realistic and complex scene images from user-specified layouts, has risen to prominence. However, existing methods transform layout information into tokens or RGB images for conditional control in the generative process, leading to insufficient spatial and semantic controllability of individual instances. To address these limitations, we propose a novel Spatial-Semantic Map Guided (SSMG) diffusion model that adopts the feature map, derived from the layout, as guidance. Owing to rich spatial and semantic information encapsulated in well-designed feature maps, SSMG achieves superior generation quality with sufficient spatial and semantic controllability compared to previous works. Additionally, we propose the Relation-Sensitive Attention (RSA) and Location-Sensitive Attention (LSA) mechanisms. The former aims to model the relationships among multiple objects within scenes while the latter is designed to heighten the model's sensitivity to the spatial information embedded in the guidance. Extensive experiments demonstrate that SSMG achieves highly promising results, setting a new state-of-the-art across a range of metrics encompassing fidelity, diversity, and controllability.

Zitieren

APA

Jia, C., Luo, M., Dang, Z., Dai, G., Chang, X., Wang, M., & Wang, J. (2024). SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-Form Layout-to-Image Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3), 2480-2488. https://doi.org/10.1609/aaai.v38i3.28024

Vancouver

Jia C, Luo M, Dang Z, Dai G, Chang X, Wang M, et al. SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-Form Layout-to-Image Generation. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(3):2480-2488. doi:10.1609/aaai.v38i3.28024

BibTeX
@article{Jia2024,
  author = {Jia, Chengyou and Luo, Minnan and Dang, Zhuohang and Dai, Guang and Chang, Xiaojun and Wang, Mengmeng and Wang, Jingdong},
  title = {SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-Form Layout-to-Image Generation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {3},
  pages = {2480--2488},
  doi = {10.1609/aaai.v38i3.28024}
}
Kostenlos starten →