EUVIMEDEUVIMED entdecken
Crossref

In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback

Mingye Zhu, Yi Liu, Zheren Fu, Quan Wang, Yongdong Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (41) · 35195-35203

DOI: 10.1609/aaai.v40i41.40826 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Mingye Zhu und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Training Large Language Models (LLMs) for chain-of-thought reasoning presents a significant challenge: supervised fine-tuning on a single "golden" rationale hurts generalization as it penalizes equally valid alternatives, whereas reinforcement learning with verifiable rewards struggles with credit assignment and prohibitive computational cost. To tackle these limitations, we introduce InTRO (In-Token Rationality Optimization), a new framework that enables both token-level exploration and self-feedback for accurate and concise reasoning. Instead of directly optimizing an intractable objective over all valid reasoning paths, InTRO leverages correction factors—token-wise importance weights estimated by the information discrepancy between the generative policy and its answer-conditioned counterpart, for informative next-token selection. This approach allows the model to perform token-level exploration and receive self-generated feedback within a single forward pass, ultimately encouraging accurate and concise rationales. Across six math-reasoning benchmarks, InTRO consistently outperforms other baselines, raising solution accuracy by up to 20% relative to the base model. Its chains of thought are also notably more concise, exhibiting reduced verbosity. Beyond this, InTRO enables cross-domain transfer, successfully adapting to out-of-domain reasoning tasks that extend beyond the realm of mathematics, demonstrating robust generalization.

Zitieren

APA

Zhu, M., Liu, Y., Fu, Z., Wang, Q., & Zhang, Y. (2026). In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41), 35195-35203. https://doi.org/10.1609/aaai.v40i41.40826

Vancouver

Zhu M, Liu Y, Fu Z, Wang Q, Zhang Y. In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(41):35195-35203. doi:10.1609/aaai.v40i41.40826

BibTeX
@article{Zhu2026,
  author = {Zhu, Mingye and Liu, Yi and Fu, Zheren and Wang, Quan and Zhang, Yongdong},
  title = {In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {41},
  pages = {35195--35203},
  doi = {10.1609/aaai.v40i41.40826}
}
Kostenlos starten →