EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Empirical Analysis of Chain-of-Thought and Solver-Augmented Large Language Models for Deductive Reasoning

Ya Wang, Raja Havish Seggoju, Adrian Paschke

Neurosymbolic Artificial Intelligence · 2026-07 · 2

DOI: 10.1177/29498732261469484 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neurosymbolic Artificial Intelligence, online erschienen am 28.07.2026. Verfasst von Ya Wang und 2 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have demonstrated remarkable capabilities in multistep reasoning, yet fundamental questions remain about their ability to perform reliable deductive reasoning. Two contrasting paradigms have emerged: chain-of-thought (CoT) prompting, which treats LLMs as self-contained reasoners, and solver-augmented approaches, which use LLMs as semantic parsers to translate problems into formal logic for symbolic execution. We present a systematic comparison of these approaches under strict single-pass evaluation across three established benchmarks (ProofWriter, PrOntoQA, and LogicalDeduction) and controlled synthetic datasets. Our findings reveal three key insights: (1) on standard benchmarks, solver augmentation yields minimal improvements ( < 1 % ) over zero-shot CoT for state-of-the-art models, suggesting diminishing performance margins between paradigms under current benchmark settings; (2) mid-sized open-source models (14B–32B) match proprietary model performance when solver-augmented, with the primary challenge shifting from generating valid syntax (small models) to accurate semantic parsing (larger models); and (3) on synthetic datasets with controlled difficulty variations, solver-augmented methods demonstrate superior robustness, maintaining 76.7% accuracy retention at extreme reasoning depths (14–17 steps) compared to 62.2% for CoT, and showing greater resilience to distracting facts and rules. These results indicate that while CoT-augmented LLMs achieve strong performance on deductive reasoning tasks up to five hops, solver augmentation remains valuable for deeper multihop deduction and for applications requiring robust and verifiable reasoning.

Zitieren

APA

Wang, Y., Seggoju, R. H., & Paschke, A. (2026). Empirical Analysis of Chain-of-Thought and Solver-Augmented Large Language Models for Deductive Reasoning. Neurosymbolic Artificial Intelligence, 2. https://doi.org/10.1177/29498732261469484

Vancouver

Wang Y, Seggoju RH, Paschke A. Empirical Analysis of Chain-of-Thought and Solver-Augmented Large Language Models for Deductive Reasoning. Neurosymbolic Artificial Intelligence. 2026;2. doi:10.1177/29498732261469484

BibTeX
@article{Wang2026,
  author = {Wang, Ya and Seggoju, Raja Havish and Paschke, Adrian},
  title = {Empirical Analysis of Chain-of-Thought and Solver-Augmented Large Language Models for Deductive Reasoning},
  journal = {Neurosymbolic Artificial Intelligence},
  year = {2026},
  volume = {2},
  doi = {10.1177/29498732261469484}
}
Kostenlos starten →