EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling

Qianhui Zhao, Li Zhang, Fang Liu, Junhang Cheng, Chengru Wu, Junchen Ai, Qiaoyuanhe Meng, Lichen Zhang, Xiaoli Lian, Shubin Song, Yuanping Guo

ACM Transactions on Software Engineering and Methodology · 30.11.2026 · 35 (11) · 1-37

DOI: 10.1145/3817056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 04.11.2026. Verfasst von Qianhui Zhao und 10 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, State Key Laboratory of Complex & Critical Software Environment.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, Large Language Models (LLMs) have achieved remarkable progress in automated code generation. In real-world software engineering, the growing demand for rapid iteration and continuous delivery underscores the importance of project-level code generation, where LLMs are expected to generate complete software projects directly from complex user requirements. This task, however, presents greater challenges due to larger input scales, multi-modal information, and the need for holistic architectural reasoning. Although existing studies have made initial explorations, they still face key limitations, including unrealistic datasets and unreliable evaluation metrics that fail to reflect real-world complexity, the semantic gap between human-written requirements and machine-interpretable structures, and difficulties in managing hierarchical dependencies and maintaining quality throughout the generation process. To address these limitations, we first introduce CodeProjectEval , a project-level code generation dataset built from 18 real-world repositories with 12.7 files and 2,388.6 lines of code per task on average, supplemented with documentation and executable test cases for automatic evaluation. We further propose ProjectGen , a multi-agent framework that decomposes projects into architecture design, skeleton generation, and code filling stages with iterative refinement and memory-based context management. Within this framework, we introduce the Semantic Software Architecture Tree, a structured and semantically rich representation that effectively bridges user requirements and source code implementation. Experiments show that ProjectGen achieves state-of-the-art performance, passing 52/124 test cases on the small-scale project-level code generation dataset DevBench, a 57% improvement over the baseline approaches, and 310 test cases on CodeProjectEval , representing an improvement of roughly tenfold compared to the baselines.

Förderung

  • National Natural Science Foundation of China
  • State Key Laboratory of Complex & Critical Software Environment

Zitieren

APA

Zhao, Q., Zhang, L., Liu, F., Cheng, J., Wu, C., Ai, J., Meng, Q., Zhang, L., Lian, X., Song, S., & Guo, Y. (2026). Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling. ACM Transactions on Software Engineering and Methodology, 35(11), 1-37. https://doi.org/10.1145/3817056

Vancouver

Zhao Q, Zhang L, Liu F, Cheng J, Wu C, Ai J, et al. Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling. ACM Transactions on Software Engineering and Methodology. 2026;35(11):1-37. doi:10.1145/3817056

BibTeX
@article{Zhao2026,
  author = {Zhao, Qianhui and Zhang, Li and Liu, Fang and Cheng, Junhang and Wu, Chengru and Ai, Junchen and Meng, Qiaoyuanhe and Zhang, Lichen and Lian, Xiaoli and Song, Shubin and Guo, Yuanping},
  title = {Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2026},
  volume = {35},
  number = {11},
  pages = {1--37},
  doi = {10.1145/3817056}
}
Kostenlos starten →