EUVIMEDEUVIMED entdecken
Crossref

Revisiting TuRTLe: A Comprehensive Evaluation of LLMs for RTL Generation

Miquel Alberti-Binimelis, Cristian Gutierrez-Gomez, Dario Garcia-Gasulla, Emanuele Parisi, Razine Moundir Ghorab, Orlando Montenegro, Bernat Homs, Miquel Moreto, Gokcen Kestor

ACM Transactions on Design Automation of Electronic Systems · 20.07.2026

DOI: 10.1145/3831369 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Design Automation of Electronic Systems, online erschienen am 20.07.2026. Verfasst von Miquel Alberti-Binimelis und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 60 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Rapid advancements in LLMs have driven the adoption of generative AI in domains like Electronic Design Automation (EDA). Within the field of software development, EDA presents unique challenges derived from specific requirements of generated RTL code; RTL code must not only be syntactically correct and functionally accurate, but also synthesizable by hardware generators, while matching performance, power and area (PPA) constraints. These additional requirements introduce complexities that existing code-generation benchmarks often fail to capture, limiting their effectiveness in evaluating LLMs for RTL generation. To address this gap, we propose TuRTLe , a unified evaluation framework designed to systematically assess LLMs across key RTL generation tasks. TuRTLe integrates multiple existing benchmarks and automates the evaluation process, enabling a comprehensive assessment of LLM performance in syntax correctness, functional correctness, synthesis, PPA optimization, and exact line completion. Using this framework, a diverse set of forty open LLMs are assesed, tracking their strengths and weaknesses in EDA-specific tasks. Our results identify the best match for specific tasks ( e.g., base models are better in module completion tasks, instruct-tuned models are better in specification-to-RTL tasks), while finding that recent models with autoregressive reasoning chain perform the best overall. We also analyze common compiler and runtime failures, study correlations between benchmarks and evaluation goals, and investigate potential training-data contamination in existing RTL datasets. These analyses provide further insight into the capabilities and limitations of current benchmarks for RTL generation.

Zitieren

APA

Alberti-Binimelis, M., Gutierrez-Gomez, C., Garcia-Gasulla, D., Parisi, E., Ghorab, R. M., Montenegro, O., Homs, B., Moreto, M., & Kestor, G. (2026). Revisiting TuRTLe: A Comprehensive Evaluation of LLMs for RTL Generation. ACM Transactions on Design Automation of Electronic Systems. https://doi.org/10.1145/3831369

Vancouver

Alberti-Binimelis M, Gutierrez-Gomez C, Garcia-Gasulla D, Parisi E, Ghorab RM, Montenegro O, et al. Revisiting TuRTLe: A Comprehensive Evaluation of LLMs for RTL Generation. ACM Transactions on Design Automation of Electronic Systems. 2026. doi:10.1145/3831369

BibTeX
@article{AlbertiBinimelis2026,
  author = {Alberti-Binimelis, Miquel and Gutierrez-Gomez, Cristian and Garcia-Gasulla, Dario and Parisi, Emanuele and Ghorab, Razine Moundir and Montenegro, Orlando and Homs, Bernat and Moreto, Miquel and Kestor, Gokcen},
  title = {Revisiting TuRTLe: A Comprehensive Evaluation of LLMs for RTL Generation},
  journal = {ACM Transactions on Design Automation of Electronic Systems},
  year = {2026},
  doi = {10.1145/3831369}
}
Kostenlos starten →