Take caution in using LLMs as human surrogates
Proceedings of the National Academy of Sciences · 17.06.2025 · 122 (24)
DOI: 10.1073/pnas.2501660122 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, online erschienen am 13.06.2025. Verfasst von Yuan Gao und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Information Systems, Questrom School of Business, Boston University, Boston, MA 02215. Hauptthemen laut MEDLINE: Language. Die Arbeit wurde bisher 58-mal zitiert (Crossref) und nennt 81 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, surveys, and political discourse. This has led many to propose that LLMs can be used as surrogates or simulations for humans in social science research. However, LLMs differ fundamentally from humans, relying on probabilistic patterns, absent the embodied experiences or survival objectives that shape human cognition. We assess the reasoning depth of LLMs using the 11-20 money request game. Nearly all advanced approaches fail to replicate human behavior distributions across many models. The causes of failure are diverse and unpredictable, relating to input language, roles, safeguarding, and more. These results warrant caution in using LLMs as surrogates or for simulating human behavior in research.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Information Systems, Questrom School of Business, Boston University, Boston, MA 02215.
- Faculty of Computing and Data Sciences, Boston University, Boston, MA 02215.
- Department of Philosophy and Religion and Khoury College of Computer Sciences, Northeastern University, Boston, MA 02115.
ORCID-Profile: Yuan Gao, Dokyun Lee, Gordon Burtch
Zitieren
APA
Gao, Y., Lee, D., Burtch, G., & Fazelpour, S. (2025). Take caution in using LLMs as human surrogates. Proceedings of the National Academy of Sciences, 122(24). https://doi.org/10.1073/pnas.2501660122
Vancouver
Gao Y, Lee D, Burtch G, Fazelpour S. Take caution in using LLMs as human surrogates. Proceedings of the National Academy of Sciences. 2025;122(24). doi:10.1073/pnas.2501660122
BibTeX
@article{Gao2025,
author = {Gao, Yuan and Lee, Dokyun and Burtch, Gordon and Fazelpour, Sina},
title = {Take caution in using LLMs as human surrogates},
journal = {Proceedings of the National Academy of Sciences},
year = {2025},
volume = {122},
number = {24},
doi = {10.1073/pnas.2501660122}
}
EUVIMED