Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30843-30851
DOI: 10.1609/aaai.v40i36.40342 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jiannan Guan und 7 weiteren Autor:innen.
Abstract (Originalsprache)
Large Language Models (LLMs) excel in reasoning tasks requiring a single correct answer, but they perform poorly in multi-solution tasks that require generating comprehensive and diverse answers. We attribute this limitation to reasoning overconfidence: a tendency to express undue certainty in an incomplete solution set. To examine the effect, we introduce MuSoBench, a benchmark of multi-solution problems. Experiments show that the conventional short chain-of-thought (Short-CoT) prompting paradigm exhibits pronounced overconfidence, whereas the emerging long chain-of-thought (Long-CoT) approach mitigates it through iterative exploration and self-reflection. We further characterise observable behaviours and influential factors. To probe the underlying cause, we propose the cognitive-rigidity hypothesis, which posits that overconfidence arises when the reasoning process prematurely converges on a narrow set of thought paths. An attention-entropy analysis offers preliminary support for this view. These findings provide tools for assessing the completeness of LLM reasoning and highlight the need to move evaluation beyond single-answer accuracy toward comprehensive exploration.
Zitieren
APA
Guan, J., Chen, Q., Qin, L., Peng, D., Liu, J., Huo, L., Xie, J., & Che, W. (2026). Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30843-30851. https://doi.org/10.1609/aaai.v40i36.40342
Vancouver
Guan J, Chen Q, Qin L, Peng D, Liu J, Huo L, et al. Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30843-30851. doi:10.1609/aaai.v40i36.40342
BibTeX
@article{Guan2026,
author = {Guan, Jiannan and Chen, Qiguang and Qin, Libo and Peng, Dengyun and Liu, Jinhao and Huo, Liangyu and Xie, Jian and Che, Wanxiang},
title = {Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {36},
pages = {30843--30851},
doi = {10.1609/aaai.v40i36.40342}
}
EUVIMED