EUVIMEDEUVIMED entdecken
Crossref

How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding

Xi Chen, Aske Plaat, Niki Van Stein

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30297-30305

DOI: 10.1609/aaai.v40i36.40281 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xi Chen und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Chain‑of‑thought (CoT) prompting boosts Large Language Models accuracy on multi‑step tasks, yet whether the generated ``thoughts'' reflect the true internal reasoning process is unresolved. We present the first feature‑level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia‑70M and Pythia‑2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT‑reasoning features into a noCoT run raises answer log‑probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear contrast for these two scales. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch‑curves and random‑feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.

Zitieren

APA

Chen, X., Plaat, A., & Stein, N. V. (2026). How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30297-30305. https://doi.org/10.1609/aaai.v40i36.40281

Vancouver

Chen X, Plaat A, Stein NV. How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30297-30305. doi:10.1609/aaai.v40i36.40281

BibTeX
@article{Chen2026,
  author = {Chen, Xi and Plaat, Aske and Stein, Niki Van},
  title = {How Does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {36},
  pages = {30297--30305},
  doi = {10.1609/aaai.v40i36.40281}
}
Kostenlos starten →