A Gold-Standard Study of What Makes a Lightweight Game-Playing Agent Strong
Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · 08.11.2026 · 22 (1) · 279-289
DOI: 10.1609/aiide.v22i1.43586 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, online erschienen am 08.11.2026. Verfasst von Nima Kelidari und 2 weiteren Autor:innen.
Abstract (Originalsprache)
A reinforcement learning agent is only as strong as the opponents it trains against, and it is also hard to grade. It beats a random opponent over 99 percent of the time, and it only ties copies of itself. So we build a strong, fixed, rule-based expert for Gin Rummy and never train against it: no expert games, no demonstrations, no gradients. It does pick the shipped checkpoint, and we say where that matters. The expert beats every agent we trained 66 to 99 percent of the time. Across more than a hundred runs we ask what makes a small agent stronger, and five things help. They are trust-region updates, a well-aimed reward, a curriculum of tougher opponents, warm starting, and keeping the best checkpoint. Stacking them lifts a self-play champion from about 30 to about 34 percent against the expert. Several other ideas did not pay off. Reward shaping at two horizons, learned state embeddings, and DAgger imitation were each unhelpful, and a live large language model opponent was far too slow. Extra capacity does little to break the ceiling, across MLP, convolutional, set-based, attention and recurrent encoders. Handing the learner the opponent's hidden hand does not move it either. We add two standard baselines: neural fictitious self-play and information-set Monte Carlo search. We also reproduce the grading protocol on Leduc Hold'em, whose optimum is computable. The result is a small, fast recipe that never trains on the expert, plus a grading protocol that carries over to any game with a fixed reference. We report bootstrap intervals on the multi-seed comparisons and release the code.
Zitieren
APA
Kelidari, N., Haghi, M., & Salmani, M. (2026). A Gold-Standard Study of What Makes a Lightweight Game-Playing Agent Strong. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 22(1), 279-289. https://doi.org/10.1609/aiide.v22i1.43586
Vancouver
Kelidari N, Haghi M, Salmani M. A Gold-Standard Study of What Makes a Lightweight Game-Playing Agent Strong. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. 2026;22(1):279-289. doi:10.1609/aiide.v22i1.43586
BibTeX
@article{Kelidari2026,
author = {Kelidari, Nima and Haghi, Mohammadsaeed and Salmani, Mahdi},
title = {A Gold-Standard Study of What Makes a Lightweight Game-Playing Agent Strong},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment},
year = {2026},
volume = {22},
number = {1},
pages = {279--289},
doi = {10.1609/aiide.v22i1.43586}
}
EUVIMED