Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling
ACM Transactions on Software Engineering and Methodology · 30.11.2026 · 35 (11) · 1-37
DOI: 10.1145/3817056 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 04.11.2026. Verfasst von Qianhui Zhao und 10 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, State Key Laboratory of Complex & Critical Software Environment.
Abstract (Originalsprache)
In recent years, Large Language Models (LLMs) have achieved remarkable progress in automated code generation. In real-world software engineering, the growing demand for rapid iteration and continuous delivery underscores the importance of project-level code generation, where LLMs are expected to generate complete software projects directly from complex user requirements. This task, however, presents greater challenges due to larger input scales, multi-modal information, and the need for holistic architectural reasoning. Although existing studies have made initial explorations, they still face key limitations, including unrealistic datasets and unreliable evaluation metrics that fail to reflect real-world complexity, the semantic gap between human-written requirements and machine-interpretable structures, and difficulties in managing hierarchical dependencies and maintaining quality throughout the generation process. To address these limitations, we first introduce CodeProjectEval , a project-level code generation dataset built from 18 real-world repositories with 12.7 files and 2,388.6 lines of code per task on average, supplemented with documentation and executable test cases for automatic evaluation. We further propose ProjectGen , a multi-agent framework that decomposes projects into architecture design, skeleton generation, and code filling stages with iterative refinement and memory-based context management. Within this framework, we introduce the Semantic Software Architecture Tree, a structured and semantically rich representation that effectively bridges user requirements and source code implementation. Experiments show that ProjectGen achieves state-of-the-art performance, passing 52/124 test cases on the small-scale project-level code generation dataset DevBench, a 57% improvement over the baseline approaches, and 310 test cases on CodeProjectEval , representing an improvement of roughly tenfold compared to the baselines.
Förderung
- National Natural Science Foundation of China
- State Key Laboratory of Complex & Critical Software Environment
Zitieren
APA
Zhao, Q., Zhang, L., Liu, F., Cheng, J., Wu, C., Ai, J., Meng, Q., Zhang, L., Lian, X., Song, S., & Guo, Y. (2026). Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling. ACM Transactions on Software Engineering and Methodology, 35(11), 1-37. https://doi.org/10.1145/3817056
Vancouver
Zhao Q, Zhang L, Liu F, Cheng J, Wu C, Ai J, et al. Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling. ACM Transactions on Software Engineering and Methodology. 2026;35(11):1-37. doi:10.1145/3817056
BibTeX
@article{Zhao2026,
author = {Zhao, Qianhui and Zhang, Li and Liu, Fang and Cheng, Junhang and Wu, Chengru and Ai, Junchen and Meng, Qiaoyuanhe and Zhang, Lichen and Lian, Xiaoli and Song, Shubin and Guo, Yuanping},
title = {Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2026},
volume = {35},
number = {11},
pages = {1--37},
doi = {10.1145/3817056}
}
EUVIMED