CATS: Category-Aware Token-level Steering for Training-Free Redundancy Reduction in Large Reasoning Models
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (41) · 34764-34771
DOI: 10.1609/aaai.v40i41.40778 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Mengfei Zhang und 1 weiteren Autor:innen.
Abstract (Originalsprache)
While Large Reasoning Models (LRMs) exhibit remarkable capabilities in complex tasks, they often suffer from excessive redundancy in their chain-of-thought reasoning. This significantly reduces inference efficiency and increases computational costs. We identify that LRM redundancy is not uniformly homogeneous but can be taxonomized according to whether it is destructive to the final answer: destructive redundancy (e.g., logical drift, hallucination amplification) versus non-destructive redundancy (e.g., repetition, over-elaboration). Moreover, LRM's redundant and concise responses exhibit a significant distinction in their hidden layer representation spaces. Based on these insights, we propose CATS (Category-Aware Token-level Steering), a training-free and lightweight method to reduce the redundancy phenomenon. CATS decomposes redundancy into six semantically interpretable characteristic dimensions. By flexibly weighting and combining the differential vectors corresponding to these dimensions, CATS synthesizes a composite intervention vector, enabling zero-parameter intervention in the hidden layers. Experiments across three LRM models and five mathematical reasoning datasets demonstrate that CATS reduces reasoning length by an average of 25% while maintaining or even slightly improving task accuracy. CATS offers a pluggable, training-free, and lightweight solution, making it particularly beneficial for users in low-resource environments.
Zitieren
APA
Zhang, M., & Wang, Z. (2026). CATS: Category-Aware Token-level Steering for Training-Free Redundancy Reduction in Large Reasoning Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41), 34764-34771. https://doi.org/10.1609/aaai.v40i41.40778
Vancouver
Zhang M, Wang Z. CATS: Category-Aware Token-level Steering for Training-Free Redundancy Reduction in Large Reasoning Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(41):34764-34771. doi:10.1609/aaai.v40i41.40778
BibTeX
@article{Zhang2026,
author = {Zhang, Mengfei and Wang, Zhenglin},
title = {CATS: Category-Aware Token-level Steering for Training-Free Redundancy Reduction in Large Reasoning Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {41},
pages = {34764--34771},
doi = {10.1609/aaai.v40i41.40778}
}
EUVIMED