EUVIMEDEUVIMED entdecken
Crossref

From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling

Zhengyu Chen, Yudong Wang, Teng Xiao, Ruochen Zhou, Xusheng Yang, Wei Wang, Zhifang Sui, Jingang Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30368-30376

DOI: 10.1609/aaai.v40i36.40289 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhengyu Chen und 7 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advancements in improving the reasoning capabilities of Large Language Models have underscored the efficacy of Process Reward Models (PRMs) in addressing intermediate errors through structured feedback mechanisms. This study analyzes PRMs from multiple perspectives, including training methodologies, scalability, and generalization capabilities. We investigate the interplay between pre-training and reward model training FLOPs to assess their influence on PRM efficiency and accuracy in complex reasoning tasks. Our analysis reveals a pattern of diminishing returns in performance with increasing PRM scale, highlighting the importance of balancing model size and computational cost. Furthermore, the diversity of training datasets significantly impacts PRM performance, emphasizing the importance of diverse data to enhance both accuracy and efficiency. We further examine test-time scaling strategies, identifying Monte Carlo Tree Search as the most effective method when computational resources are abundant, while Best-of-N Sampling serves as a practical alternative under resource-limited conditions. Notably, our findings indicate that PRMs trained on mathematical datasets exhibit performance comparable to those tailored for code generation, suggesting robust cross-domain generalization. Employing a gradient-based metric, we observe that PRMs exhibit a preference for selecting responses with similar underlying patterns, further informing their optimization.

Zitieren

APA

Chen, Z., Wang, Y., Xiao, T., Zhou, R., Yang, X., Wang, W., Sui, Z., & Wang, J. (2026). From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30368-30376. https://doi.org/10.1609/aaai.v40i36.40289

Vancouver

Chen Z, Wang Y, Xiao T, Zhou R, Yang X, Wang W, et al. From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30368-30376. doi:10.1609/aaai.v40i36.40289

BibTeX
@article{Chen2026,
  author = {Chen, Zhengyu and Wang, Yudong and Xiao, Teng and Zhou, Ruochen and Yang, Xusheng and Wang, Wei and Sui, Zhifang and Wang, Jingang},
  title = {From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {36},
  pages = {30368--30376},
  doi = {10.1609/aaai.v40i36.40289}
}
Kostenlos starten →