EUVIMEDEUVIMED entdecken
Crossref

Group Causal Policy Optimization for Post-Training Large Language Models

Ziyin Gu, Jingyao Wang, Ran Zuo, Chuxiong Sun, Zeen Song, Changwen Zheng, Wenwen Qiang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30834-30842

DOI: 10.1609/aaai.v40i36.40341 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Ziyin Gu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advances in large language models (LLMs) have broadened their applicability across diverse tasks, yet specialized domains still require targeted post-training. Among existing methods, Group Relative Policy Optimization (GRPO) stands out for its efficiency, leveraging groupwise relative rewards while avoiding costly value function learning. However, GRPO treats candidate responses as independent, overlooking semantic interactions such as complementarity and contradiction. To address this challenge, we first introduce a Structural Causal Model (SCM) that reveals hidden dependencies among candidate responses induced by conditioning on a final integrated output, forming a collider structure. Then, our causal analysis leads to two insights: (1) projecting responses onto a causally-informed subspace improves prediction quality, and (2) this projection yields a better baseline than query-only conditioning. Building on these insights, we propose Group Causal Policy Optimization (GCPO), which integrates causal structure into optimization through two key components: a causally-informed reward adjustment and a novel KL-regularization term that aligns the policy with a causally-projected reference distribution. Comprehensive experimental evaluations on various benchmarks demonstrate that GCPO consistently surpasses existing methods.

Zitieren

APA

Gu, Z., Wang, J., Zuo, R., Sun, C., Song, Z., Zheng, C., & Qiang, W. (2026). Group Causal Policy Optimization for Post-Training Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30834-30842. https://doi.org/10.1609/aaai.v40i36.40341

Vancouver

Gu Z, Wang J, Zuo R, Sun C, Song Z, Zheng C, et al. Group Causal Policy Optimization for Post-Training Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30834-30842. doi:10.1609/aaai.v40i36.40341

BibTeX
@article{Gu2026,
  author = {Gu, Ziyin and Wang, Jingyao and Zuo, Ran and Sun, Chuxiong and Song, Zeen and Zheng, Changwen and Qiang, Wenwen},
  title = {Group Causal Policy Optimization for Post-Training Large Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {36},
  pages = {30834--30842},
  doi = {10.1609/aaai.v40i36.40341}
}
Kostenlos starten →