EUVIMEDEUVIMED entdecken
Crossref

Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks

Jiannan Guan, Qiguang Chen, Libo Qin, Dengyun Peng, Jinhao Liu, Liangyu Huo, Jian Xie, Wanxiang Che

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30843-30851

DOI: 10.1609/aaai.v40i36.40342 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jiannan Guan und 7 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) excel in reasoning tasks requiring a single correct answer, but they perform poorly in multi-solution tasks that require generating comprehensive and diverse answers. We attribute this limitation to reasoning overconfidence: a tendency to express undue certainty in an incomplete solution set. To examine the effect, we introduce MuSoBench, a benchmark of multi-solution problems. Experiments show that the conventional short chain-of-thought (Short-CoT) prompting paradigm exhibits pronounced overconfidence, whereas the emerging long chain-of-thought (Long-CoT) approach mitigates it through iterative exploration and self-reflection. We further characterise observable behaviours and influential factors. To probe the underlying cause, we propose the cognitive-rigidity hypothesis, which posits that overconfidence arises when the reasoning process prematurely converges on a narrow set of thought paths. An attention-entropy analysis offers preliminary support for this view. These findings provide tools for assessing the completeness of LLM reasoning and highlight the need to move evaluation beyond single-answer accuracy toward comprehensive exploration.

Zitieren

APA

Guan, J., Chen, Q., Qin, L., Peng, D., Liu, J., Huo, L., Xie, J., & Che, W. (2026). Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30843-30851. https://doi.org/10.1609/aaai.v40i36.40342

Vancouver

Guan J, Chen Q, Qin L, Peng D, Liu J, Huo L, et al. Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30843-30851. doi:10.1609/aaai.v40i36.40342

BibTeX
@article{Guan2026,
  author = {Guan, Jiannan and Chen, Qiguang and Qin, Libo and Peng, Dengyun and Liu, Jinhao and Huo, Liangyu and Xie, Jian and Che, Wanxiang},
  title = {Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {36},
  pages = {30843--30851},
  doi = {10.1609/aaai.v40i36.40342}
}
Kostenlos starten →