EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Enhancing the Mathematical Reasoning Ability of Small Language Models through Thought Chain Distillation

Xiangyu Shu

Mathematical Modeling and Algorithm Application · 15.03.2026 · 9 (1) · 14-19

DOI: 10.54097/7cehgc40 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematical Modeling and Algorithm Application, online erschienen am 15.03.2026. Verfasst von Xiangyu Shu. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have demonstrated strong capabilities in reasoning tasks through the Chain of Thought (CoT) prompting technology, but their large scale makes it difficult to deploy them in resource-constrained environments. This paper explores the transfer of the reasoning capabilities of large models to small models through CoT Distillation to enhance the complex reasoning performance of the small models. The study uses the OpenR1-Math-220k dataset generated by DeepSeek R1 as the "teacher reasoning process" source, and employs the QLoRA parameter-efficient fine-tuning technique to train on the Qwen3-8B small model. Experimental results show that the fine-tuned model achieves an accuracy of 10% on the AIME-level mathematics test set compared to the baseline model, and can generate structured reasoning steps. The study verifies the effectiveness of CoT Distillation and provides a reproducible baseline framework for the deployment of small models in reasoning tasks. The paper hopes to provide researchers with future research directions.

Zitieren

APA

Shu, X. (2026). Enhancing the Mathematical Reasoning Ability of Small Language Models through Thought Chain Distillation. Mathematical Modeling and Algorithm Application, 9(1), 14-19. https://doi.org/10.54097/7cehgc40

Vancouver

Shu X. Enhancing the Mathematical Reasoning Ability of Small Language Models through Thought Chain Distillation. Mathematical Modeling and Algorithm Application. 2026;9(1):14-19. doi:10.54097/7cehgc40

BibTeX
@article{Shu2026,
  author = {Shu, Xiangyu},
  title = {Enhancing the Mathematical Reasoning Ability of Small Language Models through Thought Chain Distillation},
  journal = {Mathematical Modeling and Algorithm Application},
  year = {2026},
  volume = {9},
  number = {1},
  pages = {14--19},
  doi = {10.54097/7cehgc40}
}
Kostenlos starten →