EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Deductive Logic in Language Models: Horizontal vs. Vertical Reasoning

Davide Maltoni, Matteo Ferrara

Machine Learning and Knowledge Extraction · 21.07.2026 · 8 (7) · 214

DOI: 10.3390/make8070214 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 21.07.2026. Verfasst von Davide Maltoni und 1 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent language models exhibit significant logical reasoning abilities, yet the mechanisms supporting deductive inference remain poorly understood. This paper studies small transformer-based language models trained from scratch on multi-step deductive tasks, focusing on the distinction between horizontal reasoning, where intermediate steps are generated autoregressively, and vertical reasoning, where inference unfolds implicitly across layers before the first output token is produced. We analyze two synthetic tasks: logical consequence over chains of symbolic implications and root-to-leaf navigation in binary trees. Mechanistic interpretability reveals that Chain-of-Thought supervision enables models to learn rule-based inference rather than statistical shortcuts. In the horizontal setting, a shallow attention-only model develops interpretable circuits for rule completion, rule chaining, and final decision making, largely implemented through induction-head-like mechanisms. We further introduce a truncated pseudoinverse method to decode the information carried by queries, keys, and values. For vertical reasoning, Chain-of-Thought appears to act less as explicit step-by-step guidance and more as a form of curriculum learning, helping the model acquire increasingly complex reasoning patterns. Without Chain-of-Thought, models tend to memorize or exploit dataset biases. These results provide a low-level account of how transformers can implement deductive reasoning and suggest how Chain-of-Thought may serve different functions in horizontal and vertical reasoning.

Zitieren

APA

Maltoni, D., & Ferrara, M. (2026). Deductive Logic in Language Models: Horizontal vs. Vertical Reasoning. Machine Learning and Knowledge Extraction, 8(7), 214. https://doi.org/10.3390/make8070214

Vancouver

Maltoni D, Ferrara M. Deductive Logic in Language Models: Horizontal vs. Vertical Reasoning. Machine Learning and Knowledge Extraction. 2026;8(7):214. doi:10.3390/make8070214

BibTeX
@article{Maltoni2026,
  author = {Maltoni, Davide and Ferrara, Matteo},
  title = {Deductive Logic in Language Models: Horizontal vs. Vertical Reasoning},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {7},
  pages = {214},
  doi = {10.3390/make8070214}
}
Kostenlos starten →