Prototype Entropy Alignment: Reinforcing Structured Uncertainty in LLM Reasoning
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24709-24717
DOI: 10.1609/aaai.v40i29.39656 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhengyuan Pan und 6 weiteren Autor:innen.
Abstract (Originalsprache)
Recent research reveals that a minority of high-entropy tokens significantly influence the reasoning quality of large language models (LLMs). Inspired by this, we propose Prototype Entropy Alignment (PEA), a reinforcement learning framework that models effective reasoning not as a single path but as a collection of learnable "entropy signatures." PEA identifies these signatures by clustering expert trajectories' uncertainty patterns into a diverse and continuously updated set of prototypes. The model is then rewarded for aligning its own reasoning process with these evolving targets, creating a self-improvement loop. Instead of replacing traditional outcome-based rewards, PEA provides a complementary, process-oriented signal. Our experiments show that this synergy is crucial: PEA substantially boosts performance on creative and general reasoning tasks and, when combined with outcome rewards, achieves SOTA results on structured tasks such as mathematics. By rewarding alignment with diverse and evolving reasoning structures, PEA offers a robust, verifier-free pathway to enhance reasoning's adaptability.
Zitieren
APA
Pan, Z., Chen, Y., Jian, Z., Zhao, W., Ma, H., Wang, M., & Wu, Q. (2026). Prototype Entropy Alignment: Reinforcing Structured Uncertainty in LLM Reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24709-24717. https://doi.org/10.1609/aaai.v40i29.39656
Vancouver
Pan Z, Chen Y, Jian Z, Zhao W, Ma H, Wang M, et al. Prototype Entropy Alignment: Reinforcing Structured Uncertainty in LLM Reasoning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24709-24717. doi:10.1609/aaai.v40i29.39656
BibTeX
@article{Pan2026,
author = {Pan, Zhengyuan and Chen, Yanhao and Jian, Zhongquan and Zhao, Wanru and Ma, Haonan and Wang, Meihong and Wu, Qingqiang},
title = {Prototype Entropy Alignment: Reinforcing Structured Uncertainty in LLM Reasoning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {29},
pages = {24709--24717},
doi = {10.1609/aaai.v40i29.39656}
}
EUVIMED