EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating Inductive Reasoning and Programming Capabilities of Large Language Models With The One-Dimensional Abstract Reasoning Corpus

Cédric Mesnage, Xiaoyang Wang, Hang Dong, Aishwaryaprajna

Intelligenza Artificiale · 2025

DOI: 10.1177/17248035251363882 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Intelligenza Artificiale, 2025. Verfasst von Cédric Mesnage und 3 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present an initial automated test to evaluate LLMs’ capacity to perform inductive reasoning tasks. We use the GPT-3.5 and GPT-4 models to create a system which generates Python code as hypotheses for inductive reasoning to transform sequences of the One Dimensional Abstract Reasoning Corpus (1D-ARC) challenge. We experiment with three prompting techniques, namely standard prompting, Chain of Thought (CoT), and direct feedback. We provide results and an analysis of cost-to-success rate and benefit-cost ratio. Our best result is an overall 25% success rate with our CoT prompting on GPT-4, significantly surpassing the standard prompting approach. We assess the programming capabilities of the LLM by analysing the execution rate and errors of the generated code for inductive reasoning. We discuss potential avenues to improve our experiments, testing other strategies, and combining deductive reasoning with LLM-based inductive reasoning.

Zitieren

APA

Mesnage, C., Wang, X., Dong, H., & Aishwaryaprajna (2025). Evaluating Inductive Reasoning and Programming Capabilities of Large Language Models With The One-Dimensional Abstract Reasoning Corpus. Intelligenza Artificiale. https://doi.org/10.1177/17248035251363882

Vancouver

Mesnage C, Wang X, Dong H, Aishwaryaprajna. Evaluating Inductive Reasoning and Programming Capabilities of Large Language Models With The One-Dimensional Abstract Reasoning Corpus. Intelligenza Artificiale. 2025. doi:10.1177/17248035251363882

BibTeX
@article{Mesnage2025,
  author = {Mesnage, Cédric and Wang, Xiaoyang and Dong, Hang and Aishwaryaprajna},
  title = {Evaluating Inductive Reasoning and Programming Capabilities of Large Language Models With The One-Dimensional Abstract Reasoning Corpus},
  journal = {Intelligenza Artificiale},
  year = {2025},
  doi = {10.1177/17248035251363882}
}
Kostenlos starten →