ControlScene: Controllable Text‐to‐3D Scene Generation via Structured Layout Priors
CAAI Transactions on Intelligence Technology · 09.11.2026
DOI: 10.1049/cit2.70192 ↗Kurz gefasst
Fachartikel in der Zeitschrift CAAI Transactions on Intelligence Technology, online erschienen am 09.11.2026. Verfasst von Ruyi Zhang und 5 weiteren Autor:innen. Die Arbeit nennt 81 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Natural Science Foundation of China, National Key Research and Development Program of China.
Abstract (Originalsprache)
Text‐to‐3D scene generation has emerged as a promising approach for enabling natural and intuitive 3D content creation through language. However, existing methods often struggle with generating scenes that are semantically faithful, spatially coherent and controllable, primarily due to the lack of grounded spatial reasoning and fine‐grained structural supervision. In this paper, we present ControlScene: controllable text‐to‐3D scene generation via structured layout priors, a novel framework that incorporates layout‐level structure into the generation process, allowing LLM to jointly reason about semantics and spatial composition. To support this framework, we introduce LayoutVerse‐20K, a large‐scale benchmark dataset comprising 20,000 manually annotated samples, each including textual prompts, layout metadata, graph‐structured layouts, scene graphs and corresponding 3D scenes. For fine‐grained evaluation, we propose two task‐specific metrics: category and layout plausibility, which assess semantic and spatial alignment by comparing generated outputs against common sense priors derived from top‐k reference samples. Additionally, we develop an interactive user interface for real‐time layout customisation, enhancing controllability and practical usability. Extensive experiments show that ControlScene significantly outperforms baseline methods in spatial realism, semantic consistency and user controllability, providing a robust foundation for future research in grounded language‐driven 3D scene generation. Project site: https://winkquan.github.io/ControlScene‐Page .
Förderung
- National Natural Science Foundation of China
- National Key Research and Development Program of China
Zitieren
APA
Zhang, R., Zhou, Z., Quan, W., Li, W., Zuo, W., & Fan, X. (2026). ControlScene: Controllable Text‐to‐3D Scene Generation via Structured Layout Priors. CAAI Transactions on Intelligence Technology. https://doi.org/10.1049/cit2.70192
Vancouver
Zhang R, Zhou Z, Quan W, Li W, Zuo W, Fan X. ControlScene: Controllable Text‐to‐3D Scene Generation via Structured Layout Priors. CAAI Transactions on Intelligence Technology. 2026. doi:10.1049/cit2.70192
BibTeX
@article{Zhang2026,
author = {Zhang, Ruyi and Zhou, Zhi and Quan, Wenke and Li, Wenrui and Zuo, Wangmeng and Fan, Xiaopeng},
title = {ControlScene: Controllable Text‐to‐3D Scene Generation via Structured Layout Priors},
journal = {CAAI Transactions on Intelligence Technology},
year = {2026},
doi = {10.1049/cit2.70192}
}
EUVIMED