Benchmarking LLMs for Political Science: A United Nations Perspective
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (1) · 738-745
DOI: 10.1609/aaai.v40i1.37040 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yueqing Liang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
Large Language Models (LLMs) have achieved significant advances in natural language processing, yet their potential for high-stake political decision-making remains largely unexplored. This paper addresses the gap by focusing on the application of LLMs to the United Nations (UN) decision-making process, where the stakes are particularly high and political decisions can have far-reaching consequences. We introduce a novel dataset comprising publicly available UN Security Council (UNSC) records from 1994 to 2024, including draft resolutions, voting records, and diplomatic speeches. Using this dataset, we propose the United Nations Benchmark (UNBench), the first comprehensive benchmark designed to evaluate LLMs across four interconnected political science tasks: co-penholder judgment, representative voting simulation, draft adoption prediction, and representative statement generation. These tasks span the three stages of the UN decision-making process—drafting, voting, and discussing—and aim to assess LLMs' ability to understand and simulate political dynamics. Our experimental analysis demonstrates the potential and challenges of applying LLMs in this domain, providing insights into their strengths and limitations in political science. To the best of our knowledge, this is the first benchmark to systematically evaluate LLMs in UN decision-making, contributing to the growing intersection of AI and political science.
Zitieren
APA
Liang, Y., Yang, L., Wang, C., Xia, C., Meng, R., Xu, X., Wang, H., Payani, A., & Shu, K. (2026). Benchmarking LLMs for Political Science: A United Nations Perspective. Proceedings of the AAAI Conference on Artificial Intelligence, 40(1), 738-745. https://doi.org/10.1609/aaai.v40i1.37040
Vancouver
Liang Y, Yang L, Wang C, Xia C, Meng R, Xu X, et al. Benchmarking LLMs for Political Science: A United Nations Perspective. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(1):738-745. doi:10.1609/aaai.v40i1.37040
BibTeX
@article{Liang2026,
author = {Liang, Yueqing and Yang, Liangwei and Wang, Chen and Xia, Congying and Meng, Rui and Xu, Xiongxiao and Wang, Haoran and Payani, Ali and Shu, Kai},
title = {Benchmarking LLMs for Political Science: A United Nations Perspective},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {1},
pages = {738--745},
doi = {10.1609/aaai.v40i1.37040}
}
EUVIMED