When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract)
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (48) · 41228-41230
DOI: 10.1609/aaai.v40i48.42223 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Azka Ikramullah und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Large language models (LLMs) often perform better when prompted to explain their reasoning, but it remains unclear how well such gains persist as reasoning depth increases. In this work, we propose a depth-aware evaluation framework alongside the performance results on two structured datasets: CLUTRR (kinship reasoning) and ProofWriter (logical entailment), comparing direct vs. reasoning (reasoning depth = number of inference steps required) prompts across five models. Reasoning gave small gains at shallow depths but quickly weakened and often reversed as tasks grew more complex. In ProofWriter, GPT-5 reached 90% accuracy at depth four in direct model, yet its reasoning accuracy fell below baseline after depth two. Smaller open-source models showed only unstable or negligible gains, underscoring that reasoning in LLMs remains brittle with increased depth.
Zitieren
APA
Ikramullah, A., Majeed, A., Lee, K., & Hwang, S. O. (2026). When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 40(48), 41228-41230. https://doi.org/10.1609/aaai.v40i48.42223
Vancouver
Ikramullah A, Majeed A, Lee K, Hwang SO. When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(48):41228-41230. doi:10.1609/aaai.v40i48.42223
BibTeX
@article{Ikramullah2026,
author = {Ikramullah, Azka and Majeed, Abdul and Lee, Kyunghyun and Hwang, Seong Oun},
title = {When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract)},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {48},
pages = {41228--41230},
doi = {10.1609/aaai.v40i48.42223}
}
EUVIMED