EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic Programming

Ning Tao, Anthony Ventresque, Vivek Nallur, Takfarinas Saber

Algorithms · 01.07.2024 · 17 (7) · 287

DOI: 10.3390/a17070287 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Algorithms, online erschienen am 01.07.2024. Verfasst von Ning Tao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Science Foundation Ireland.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The ability to automatically generate code, i.e., program synthesis, is one of the most important applications of artificial intelligence (AI). Currently, two AI techniques are leading the way: large language models (LLMs) and genetic programming (GP) methods—each with its strengths and weaknesses. While LLMs have shown success in program synthesis from a task description, they often struggle to generate the correct code due to ambiguity in task specifications, complex programming syntax, and lack of reliability in the generated code. Furthermore, their generative nature limits their ability to fix erroneous code with iterative LLM prompting. Grammar-guided genetic programming (G3P, i.e., one of the top GP methods) has been shown capable of evolving programs that fit a defined Backus–Naur-form (BNF) grammar based on a set of input/output tests that help guide the search process while ensuring that the generated code does not include calls to untrustworthy libraries or poorly structured snippets. However, G3P still faces issues generating code for complex tasks. A recent study attempting to combine both approaches (G3P and LLMs) by seeding an LLM-generated program into the initial population of the G3P has shown promising results. However, the approach rapidly loses the seeded information over the evolutionary process, which hinders its performance. In this work, we propose combining an LLM (specifically ChatGPT) with a many-objective G3P (MaOG3P) framework in two parts: (i) provide the LLM-generated code as a seed to the evolutionary process following a grammar-mapping phase that creates an avenue for program evolution and error correction; and (ii) leverage many-objective similarity measures towards the LLM-generated code to guide the search process throughout the evolution. The idea behind using the similarity measures is that the LLM-generated code is likely to be close to the correct fitting code. Our approach compels any generated program to adhere to the BNF grammar, ultimately mitigating security risks and improving code quality. Experiments on a well-known and widely used program synthesis dataset show that our approach successfully improves the synthesis of grammar-fitting code for several tasks.

Förderung

  • Science Foundation Ireland

Zitieren

APA

Tao, N., Ventresque, A., Nallur, V., & Saber, T. (2024). Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic Programming. Algorithms, 17(7), 287. https://doi.org/10.3390/a17070287

Vancouver

Tao N, Ventresque A, Nallur V, Saber T. Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic Programming. Algorithms. 2024;17(7):287. doi:10.3390/a17070287

BibTeX
@article{Tao2024,
  author = {Tao, Ning and Ventresque, Anthony and Nallur, Vivek and Saber, Takfarinas},
  title = {Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic Programming},
  journal = {Algorithms},
  year = {2024},
  volume = {17},
  number = {7},
  pages = {287},
  doi = {10.3390/a17070287}
}
Kostenlos starten →