EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

VGPO-MCTS: Distilling Step-Level Supervision from Value-Guided Tree Search for Mathematical Reasoning

Pin Wu, Yufei Zhu, Huiyan Wang

AI · 17.04.2026 · 7 (4) · 146

DOI: 10.3390/ai7040146 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI, online erschienen am 17.04.2026. Verfasst von Pin Wu und 2 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) are increasingly used in applied intelligent systems, but mid-sized models still lag on mathematical reasoning, partly because reliable step-level supervision is scarce. Many existing remedies rely on costly human annotation, stronger teacher models, or heavy training pipelines, which limits practical adoption. We propose VGPO-MCTS (Value-Guided Group-wise Policy Optimization over Monte Carlo Tree Search), a search-and-distillation framework that constructs reusable step-level supervision from datasets that provide only problems and final answers. VGPO-MCTS augments a frozen backbone with (i) a lightweight value model that scores candidate reasoning states formed by a reasoning prefix and its candidate next step, and (ii) a policy updated with parameter-efficient adaptation. During search, the value model guides tree expansion and selection, while verified outcomes are propagated backward to correct node utilities. The corrected search trees are then distilled into two complementary datasets: a value regression dataset for value learning and group-wise sibling candidate sets for GRPO-style policy optimization. Experiments on GSM8K and the MATH dataset with ChatGLM3-6B and SciGLM-6B show stable round-wise improvements in final-answer exact match under a lightweight adaptation setting. After three rounds of self-training, the proposed framework improves performance by about 6.3 percentage points on GSM8K and about 3.9 percentage points on MATH across the two backbones.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Wu, P., Zhu, Y., & Wang, H. (2026). VGPO-MCTS: Distilling Step-Level Supervision from Value-Guided Tree Search for Mathematical Reasoning. AI, 7(4), 146. https://doi.org/10.3390/ai7040146

Vancouver

Wu P, Zhu Y, Wang H. VGPO-MCTS: Distilling Step-Level Supervision from Value-Guided Tree Search for Mathematical Reasoning. AI. 2026;7(4):146. doi:10.3390/ai7040146

BibTeX
@article{Wu2026,
  author = {Wu, Pin and Zhu, Yufei and Wang, Huiyan},
  title = {VGPO-MCTS: Distilling Step-Level Supervision from Value-Guided Tree Search for Mathematical Reasoning},
  journal = {AI},
  year = {2026},
  volume = {7},
  number = {4},
  pages = {146},
  doi = {10.3390/ai7040146}
}
Kostenlos starten →