LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (44) · 37802-37809
DOI: 10.1609/aaai.v40i44.41116 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Matthew Lyle Olson und 7 weiteren Autor:innen.
Abstract (Originalsprache)
Large Language Models (LLMs) exhibit impressive general-purpose capabilities but also introduce serious safety risks, particularly the potential for deception as models acquire increased agency and human oversight diminishes. In this work, we present LieCraft: a novel evaluation framework and sandbox for measuring LLM deception that addresses key limitations of prior game-based evaluations. At its core, LieCraft is a novel multiplayer hidden-role game in which players select an ethical alignment and execute strategies over a long time-horizon to accomplish missions. Cooperators work together to solve event challenges and expose bad actors, while Defectors evade suspicion while secretly sabotaging missions. To enable real-world relevance, we develop 10 grounded scenarios such as childcare, hospital resource allocation, and loan underwriting that recontextualize the underlying mechanics in ethically significant, high-stakes domains. We ensure balanced gameplay in LieCraft through careful design of game mechanics and reward structures that incentivize meaningful strategic choices while eliminating degenerate strategies. Beyond the framework itself, we report results from 12 state-of-the-art LLMs across three behavioral axes: propensity to defect, deception skill, and accusation accuracy. Our findings reveal that despite differences in competence and overall alignment, all models are willing to act unethically, conceal their intentions, and outright lie to pursue their goals.
Zitieren
APA
Olson, M. L., Ratzlaff, N., Hinck, M., Nguyen, T., Lal, V., Campbell, J., Stepputtis, S., & Tseng, S. (2026). LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(44), 37802-37809. https://doi.org/10.1609/aaai.v40i44.41116
Vancouver
Olson ML, Ratzlaff N, Hinck M, Nguyen T, Lal V, Campbell J, et al. LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(44):37802-37809. doi:10.1609/aaai.v40i44.41116
BibTeX
@article{Olson2026,
author = {Olson, Matthew Lyle and Ratzlaff, Neale and Hinck, Musashi and Nguyen, Tri and Lal, Vasudev and Campbell, Joseph and Stepputtis, Simon and Tseng, Shao-Yen},
title = {LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {44},
pages = {37802--37809},
doi = {10.1609/aaai.v40i44.41116}
}
EUVIMED