Enhanced Policy Update Mechanism for PPO-Inspired Algorithms in Intelligent Agent-Based Games
IEEE Transactions on Games · 2026-06 · 18 (2) · 340-348
DOI: 10.1109/tg.2026.3683937 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Games, 2026-06. Verfasst von Xuyan Jin und 4 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben. Gefördert von: Department of Education of Liaoning Province.
Abstract (Originalsprache)
In this article, we propose Kullback–Leibler-Beyond-Clip proximal policy optimization (KLBC-PPO), a novel algorithm derived from PPO, designed to offer a more efficient policy update mechanism. The PPO-Clip algorithm limits the exploration of new policies by truncating them beyond predefined intervals, which may hinder the identification of optimal solutions. KLBC-PPO retains the PPO-Clip method for updating parameters within the clipping range and incorporates a KL penalty for updates that exceed the range, thereby enhancing exploration while preserving efficiency. We evaluated the new algorithm in three distinct game environments: Overcooked (discrete action space), Moving-v0 (hybrid action space), and Gfootball (multiagent). The results demonstrate that KLBC-PPO outperformed PPO in Overcooked, KLBC-hybrid proximal policy optimization (HPPO) achieved superior performance earlier than HPPO in Moving-v0, and KLBC-multiagent proximal policy optimization (MAPPO) exhibited better performance in Gfootball, surpassing MAPPO. Overall, the experimental results indicate that KLBC enhances the algorithm's performance while maintaining efficiency and demonstrating strong generalization capabilities.
Förderung
- Department of Education of Liaoning Province
Zitieren
APA
Jin, X., Cao, Y., Yu, H., Deng, H., & Chen, X. (2026). Enhanced Policy Update Mechanism for PPO-Inspired Algorithms in Intelligent Agent-Based Games. IEEE Transactions on Games, 18(2), 340-348. https://doi.org/10.1109/tg.2026.3683937
Vancouver
Jin X, Cao Y, Yu H, Deng H, Chen X. Enhanced Policy Update Mechanism for PPO-Inspired Algorithms in Intelligent Agent-Based Games. IEEE Transactions on Games. 2026;18(2):340-348. doi:10.1109/tg.2026.3683937
BibTeX
@article{Jin2026,
author = {Jin, Xuyan and Cao, Yang and Yu, Hongfei and Deng, Hang and Chen, Xuejian},
title = {Enhanced Policy Update Mechanism for PPO-Inspired Algorithms in Intelligent Agent-Based Games},
journal = {IEEE Transactions on Games},
year = {2026},
volume = {18},
number = {2},
pages = {340--348},
doi = {10.1109/tg.2026.3683937}
}
EUVIMED