VerilogASTBench: Benchmark Construction of Verilog AST Dataset with Dual-Stage AST Semantic Enhancement Framework
Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 989-1010
DOI: 10.1145/3797073 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Luping Zhang und 7 weiteren Autor:innen. Die Arbeit nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Research Start-up Foundation of Recruiting Talents of Nanjing University of Posts and Telecommunications, National Natural Science Foundation of China, Self-developed Project of Nanjing University of Posts and Telecommunications Nantong Research Institute Co., Ltd..
Abstract (Originalsprache)
With the increasing complexity and scale of integrated circuit (IC) designs, automated circuit design methods are essential for Verilog implementation. Although Large Language Models (LLMs) perform well in general-purpose coding such as C++ and Python, their performance in Verilog is constrained by domain-specific semantic and structural rules as well as the scarcity of high-quality training datasets. To address these issues, this work proposes a semantically enhanced Verilog code parsing and automatic repair framework based on Abstract Syntax Tree (AST). First, an advanced register-transfer level (RTL) analysis framework was developed to achieve semantic enhancement through static analysis–driven functional role inference and attribute annotation. Second, enhanced AST information is used to construct structured prompts and generate semantically rich module descriptions via an Application Programming Interface (API) for dataset construction. Finally, an AST-guided automatic Verilog repair framework was designed, which leverages enhanced AST analysis for precise defect localization and intelligent repair through compiler feedback loops. Experimental results indicate that the proposed method successfully repaired 15.24% of defective Verilog code, resulting in a high-quality RTL benchmark dataset containing 318,021 samples. Models fine-tuned on this dataset demonstrate significant performance improvements across three benchmarks, achieving average improvements of 13.76% and 16.95% on Eval-Machine pass@1 and pass@5, 8.97% and 14.56% on Eval-Human pass@1 and pass@5, and 15.70% and 12.20% on RTLLM V1.1 Syntax-VCS and Func.
Förderung
- Natural Science Research Start-up Foundation of Recruiting Talents of Nanjing University of Posts and Telecommunications
- National Natural Science Foundation of China
- Self-developed Project of Nanjing University of Posts and Telecommunications Nantong Research Institute Co., Ltd.
Zitieren
APA
Zhang, L., Chen, C., Yan, D., Xu, H., Cao, M., Song, J., Cai, Z., & Guo, Y. (2026). VerilogASTBench: Benchmark Construction of Verilog AST Dataset with Dual-Stage AST Semantic Enhancement Framework. Proceedings of the ACM on Software Engineering, 3(FSE), 989-1010. https://doi.org/10.1145/3797073
Vancouver
Zhang L, Chen C, Yan D, Xu H, Cao M, Song J, et al. VerilogASTBench: Benchmark Construction of Verilog AST Dataset with Dual-Stage AST Semantic Enhancement Framework. Proceedings of the ACM on Software Engineering. 2026;3(FSE):989-1010. doi:10.1145/3797073
BibTeX
@article{Zhang2026,
author = {Zhang, Luping and Chen, Chao and Yan, Dapeng and Xu, Hui and Cao, Mingsheng and Song, Jingkuan and Cai, Zhikuang and Guo, Yufeng},
title = {VerilogASTBench: Benchmark Construction of Verilog AST Dataset with Dual-Stage AST Semantic Enhancement Framework},
journal = {Proceedings of the ACM on Software Engineering},
year = {2026},
volume = {3},
number = {FSE},
pages = {989--1010},
doi = {10.1145/3797073}
}
EUVIMED