EUVIMEDEUVIMED entdecken
Crossref

When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract)

Azka Ikramullah, Abdul Majeed, Kyunghyun Lee, Seong Oun Hwang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (48) · 41228-41230

DOI: 10.1609/aaai.v40i48.42223 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Azka Ikramullah und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) often perform better when prompted to explain their reasoning, but it remains unclear how well such gains persist as reasoning depth increases. In this work, we propose a depth-aware evaluation framework alongside the performance results on two structured datasets: CLUTRR (kinship reasoning) and ProofWriter (logical entailment), comparing direct vs. reasoning (reasoning depth = number of inference steps required) prompts across five models. Reasoning gave small gains at shallow depths but quickly weakened and often reversed as tasks grew more complex. In ProofWriter, GPT-5 reached 90% accuracy at depth four in direct model, yet its reasoning accuracy fell below baseline after depth two. Smaller open-source models showed only unstable or negligible gains, underscoring that reasoning in LLMs remains brittle with increased depth.

Zitieren

APA

Ikramullah, A., Majeed, A., Lee, K., & Hwang, S. O. (2026). When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 40(48), 41228-41230. https://doi.org/10.1609/aaai.v40i48.42223

Vancouver

Ikramullah A, Majeed A, Lee K, Hwang SO. When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(48):41228-41230. doi:10.1609/aaai.v40i48.42223

BibTeX
@article{Ikramullah2026,
  author = {Ikramullah, Azka and Majeed, Abdul and Lee, Kyunghyun and Hwang, Seong Oun},
  title = {When Reasoning Collapses: A Depth-Aware Probe into LLM Reasoning (Student Abstract)},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {48},
  pages = {41228--41230},
  doi = {10.1609/aaai.v40i48.42223}
}
Kostenlos starten →