How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30297-30305
DOI: 10.1609/aaai.v40i36.40281 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xi Chen und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Chain‑of‑thought (CoT) prompting boosts Large Language Models accuracy on multi‑step tasks, yet whether the generated ``thoughts'' reflect the true internal reasoning process is unresolved. We present the first feature‑level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia‑70M and Pythia‑2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT‑reasoning features into a noCoT run raises answer log‑probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear contrast for these two scales. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch‑curves and random‑feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.
Zitieren
APA
Chen, X., Plaat, A., & Stein, N. V. (2026). How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30297-30305. https://doi.org/10.1609/aaai.v40i36.40281
Vancouver
Chen X, Plaat A, Stein NV. How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30297-30305. doi:10.1609/aaai.v40i36.40281
BibTeX
@article{Chen2026,
author = {Chen, Xi and Plaat, Aske and Stein, Niki Van},
title = {How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {36},
pages = {30297--30305},
doi = {10.1609/aaai.v40i36.40281}
}
EUVIMED