Imitation-Initialized Reinforcement Learning for Strategic Decision-Making In Settlers of Catan
Studia Universitatis Babeș-Bolyai Informatica · 12.11.2026 · 71 (1-2) · 58-72
DOI: 10.24193/subbi.2026.04 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studia Universitatis Babeș-Bolyai Informatica, online erschienen am 12.11.2026. Verfasst von Sorin-Nicolae TURCULET. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
This paper studies how to train a reinforcement learning agent for Settlers of Catan, a board game that is difficult to learn because of dice randomness, hidden opponent cards, a large action space, and sparse win/loss rewards at the end of a long game. We apply Proximal Policy Optimization (PPO) and compare three ways of training the same agent: Pure-PPO (learning from scratch with no external help), Expert- Guided PPO (EG-PPO) adding an online reward bonus whenever the agent picks a move favored by a heuristic expert), and BC-PPO (first copying an expert via Behavioral Cloning and then fine-tuning freely with reinforcement learning). Pure-PPO never exceeds a 10% win rate against a strong heuristic baseline regardless of architecture or reward shaping. EGPPO lifts this ceiling to 31.2%, and BC-PPO reaches 69.6%, more than doubling the best reward-shaping result. We also report a counter-intuitive teacher-quality paradox: cloning a stronger multi-ply Alpha-Beta search expert yields a worse final agent than cloning a simpler 1-ply heuristic, because the reactive network cannot observe the search tree that motivated the stronger teacher’s moves.
Zitieren
APA
TURCULET, S. (2026). Imitation-Initialized Reinforcement Learning for Strategic Decision-Making In Settlers of Catan. Studia Universitatis Babeș-Bolyai Informatica, 71(1-2), 58-72. https://doi.org/10.24193/subbi.2026.04
Vancouver
TURCULET S. Imitation-Initialized Reinforcement Learning for Strategic Decision-Making In Settlers of Catan. Studia Universitatis Babeș-Bolyai Informatica. 2026;71(1-2):58-72. doi:10.24193/subbi.2026.04
BibTeX
@article{TURCULET2026,
author = {TURCULET, Sorin-Nicolae},
title = {Imitation-Initialized Reinforcement Learning for Strategic Decision-Making In Settlers of Catan},
journal = {Studia Universitatis Babeș-Bolyai Informatica},
year = {2026},
volume = {71},
number = {1-2},
pages = {58--72},
doi = {10.24193/subbi.2026.04}
}
EUVIMED