EUVIMEDEUVIMED entdecken
Crossref

LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models

Matthew Lyle Olson, Neale Ratzlaff, Musashi Hinck, Tri Nguyen, Vasudev Lal, Joseph Campbell, Simon Stepputtis, Shao-Yen Tseng

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (44) · 37802-37809

DOI: 10.1609/aaai.v40i44.41116 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Matthew Lyle Olson und 7 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) exhibit impressive general-purpose capabilities but also introduce serious safety risks, particularly the potential for deception as models acquire increased agency and human oversight diminishes. In this work, we present LieCraft: a novel evaluation framework and sandbox for measuring LLM deception that addresses key limitations of prior game-based evaluations. At its core, LieCraft is a novel multiplayer hidden-role game in which players select an ethical alignment and execute strategies over a long time-horizon to accomplish missions. Cooperators work together to solve event challenges and expose bad actors, while Defectors evade suspicion while secretly sabotaging missions. To enable real-world relevance, we develop 10 grounded scenarios such as childcare, hospital resource allocation, and loan underwriting that recontextualize the underlying mechanics in ethically significant, high-stakes domains. We ensure balanced gameplay in LieCraft through careful design of game mechanics and reward structures that incentivize meaningful strategic choices while eliminating degenerate strategies. Beyond the framework itself, we report results from 12 state-of-the-art LLMs across three behavioral axes: propensity to defect, deception skill, and accusation accuracy. Our findings reveal that despite differences in competence and overall alignment, all models are willing to act unethically, conceal their intentions, and outright lie to pursue their goals.

Zitieren

APA

Olson, M. L., Ratzlaff, N., Hinck, M., Nguyen, T., Lal, V., Campbell, J., Stepputtis, S., & Tseng, S. (2026). LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(44), 37802-37809. https://doi.org/10.1609/aaai.v40i44.41116

Vancouver

Olson ML, Ratzlaff N, Hinck M, Nguyen T, Lal V, Campbell J, et al. LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(44):37802-37809. doi:10.1609/aaai.v40i44.41116

BibTeX
@article{Olson2026,
  author = {Olson, Matthew Lyle and Ratzlaff, Neale and Hinck, Musashi and Nguyen, Tri and Lal, Vasudev and Campbell, Joseph and Stepputtis, Simon and Tseng, Shao-Yen},
  title = {LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {44},
  pages = {37802--37809},
  doi = {10.1609/aaai.v40i44.41116}
}
Kostenlos starten →