EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Take caution in using LLMs as human surrogates

Yuan Gao iD, Dokyun Lee iD, Gordon Burtch iD, Sina Fazelpour

Proceedings of the National Academy of Sciences · 17.06.2025 · 122 (24)

DOI: 10.1073/pnas.2501660122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, online erschienen am 13.06.2025. Verfasst von Yuan Gao und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Information Systems, Questrom School of Business, Boston University, Boston, MA 02215. Hauptthemen laut MEDLINE: Language. Die Arbeit wurde bisher 58-mal zitiert (Crossref) und nennt 81 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent studies suggest large language models (LLMs) can generate human-like responses, aligning with human behavior in economic experiments, surveys, and political discourse. This has led many to propose that LLMs can be used as surrogates or simulations for humans in social science research. However, LLMs differ fundamentally from humans, relying on probabilistic patterns, absent the embodied experiences or survival objectives that shape human cognition. We assess the reasoning depth of LLMs using the 11-20 money request game. Nearly all advanced approaches fail to replicate human behavior distributions across many models. The causes of failure are diverse and unpredictable, relating to input language, roles, safeguarding, and more. These results warrant caution in using LLMs as surrogates or for simulating human behavior in research.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LanguageHumansCognition

Stichwörter der Autor:innen

LLMs as a simulationLLMs as human surrogatesLLMs in social science researchSimulations of human behavior

Autor:innen und Institutionen

  • Department of Information Systems, Questrom School of Business, Boston University, Boston, MA 02215.
  • Faculty of Computing and Data Sciences, Boston University, Boston, MA 02215.
  • Department of Philosophy and Religion and Khoury College of Computer Sciences, Northeastern University, Boston, MA 02115.

ORCID-Profile: Yuan Gao, Dokyun Lee, Gordon Burtch

Zitieren

APA

Gao, Y., Lee, D., Burtch, G., & Fazelpour, S. (2025). Take caution in using LLMs as human surrogates. Proceedings of the National Academy of Sciences, 122(24). https://doi.org/10.1073/pnas.2501660122

Vancouver

Gao Y, Lee D, Burtch G, Fazelpour S. Take caution in using LLMs as human surrogates. Proceedings of the National Academy of Sciences. 2025;122(24). doi:10.1073/pnas.2501660122

BibTeX
@article{Gao2025,
  author = {Gao, Yuan and Lee, Dokyun and Burtch, Gordon and Fazelpour, Sina},
  title = {Take caution in using LLMs as human surrogates},
  journal = {Proceedings of the National Academy of Sciences},
  year = {2025},
  volume = {122},
  number = {24},
  doi = {10.1073/pnas.2501660122}
}
Kostenlos starten →