T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-Image Generation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-05 · 47 (5) · 3563-3579
DOI: 10.1109/tpami.2025.3531907 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-05. Verfasst von Kaiyi Huang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 59-mal zitiert (Crossref) und nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, HKU IDS research Seed Fund, HKU Fintech Academy R&D Funding.
Abstract (Originalsprache)
Despite the impressive advances in text-to-image models, they often struggle to effectively compose complex scenes with multiple objects, displaying various attributes and relationships. To address this challenge, we present T2I-CompBench++, an enhanced benchmark for compositional text-to-image generation. T2I-CompBench++ comprises 8,000 compositional text prompts categorized into four primary groups: attribute binding, object relationships, generative numeracy, and complex compositions. These are further divided into eight sub-categories, including newly introduced ones like 3D-spatial relationships and numeracy. In addition to the benchmark, we propose enhanced evaluation metrics designed to assess these diverse compositional challenges. These include a detection-based metric tailored for evaluating 3D-spatial relationships and numeracy, and an analysis leveraging Multimodal Large Language Models (MLLMs), i.e. GPT-4 V, ShareGPT4v as evaluation metrics. Our experiments benchmark 11 text-to-image models, including state-of-the-art models, such as FLUX.1, SD3, DALLE-3, Pixart-$\alpha$α, and SD-XL on T2I-CompBench++. We also conduct comprehensive evaluations to validate the effectiveness of our metrics and explore the potential and limitations of MLLMs.
Förderung
- National Natural Science Foundation of China
- HKU IDS research Seed Fund
- HKU Fintech Academy R&D Funding
- HKU Seed Fund for Basic Research
- HKU Seed Fund for Translational and Applied Research
Zitieren
APA
Huang, K., Duan, C., Sun, K., Xie, E., Li, Z., & Liu, X. (2025). T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-Image Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5), 3563-3579. https://doi.org/10.1109/tpami.2025.3531907
Vancouver
Huang K, Duan C, Sun K, Xie E, Li Z, Liu X. T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-Image Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(5):3563-3579. doi:10.1109/tpami.2025.3531907
BibTeX
@article{Huang2025,
author = {Huang, Kaiyi and Duan, Chengqi and Sun, Kaiyue and Xie, Enze and Li, Zhenguo and Liu, Xihui},
title = {T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-Image Generation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {5},
pages = {3563--3579},
doi = {10.1109/tpami.2025.3531907}
}
EUVIMED