EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

The potential existential threat of large language models to online survey research

Sean J Westwood iD

Proceedings of the National Academy of Sciences · 25.11.2025 · 122 (47)

DOI: 10.1073/pnas.2518075122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, online erschienen am 20.11.2025. Verfasst von Sean J Westwood. Beteiligte Einrichtung: Department of Government, Dartmouth College, Hanover, NH 03755. Hauptthemen laut MEDLINE: Language, Internet. Die Arbeit wurde bisher 95-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The advancement of large language models poses a severe, potentially existential threat to online survey research, a fundamental tool for data collection across the sciences. This work demonstrates that the foundational assumption of survey research—that a coherent response is a human response—is no longer tenable. I designed and tested an autonomous synthetic respondent capable of producing survey data that possesses the coherence and plausibility of human responses. This agent successfully evades a comprehensive suite of data quality checks, including instruction-following tasks, logic puzzles, and “reverse shibboleth” questions designed to detect nonhuman actors, achieving a 99.8% pass rate on 6,000 trials of standard attention checks. The synthetic respondent generates internally consistent responses by maintaining a coherent demographic persona and a memory of its prior answers, producing plausible data on psychometric scales, vignette comprehension tasks, and complex socioeconomic trade-offs. Furthermore, its open-ended text responses are linguistically sophisticated and stylistically calibrated to the level of education of its assigned persona. Critically, the agent can be instructed to maliciously alter polling outcomes, demonstrating an overt vector for information warfare. More subtly, it can also infer a researcher’s latent hypotheses and produce data that artificially confirms them. These findings reveal a critical vulnerability in our data infrastructure, rendering most current detection methods obsolete and posing a potential existential threat to unsupervised online research. The scientific community must urgently develop new data validation standards and reconsider its reliance on nonprobability, low-barrier online data collection methods.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LanguageInternetHumansSurveys and QuestionnairesLarge Language Models

Stichwörter der Autor:innen

large language modelssurvey qualitysurveys

Autor:innen und Institutionen

  • Department of Government, Dartmouth College, Hanover, NH 03755.

ORCID-Profile: Sean J Westwood

Zitieren

APA

Westwood, S. J. (2025). The potential existential threat of large language models to online survey research. Proceedings of the National Academy of Sciences, 122(47). https://doi.org/10.1073/pnas.2518075122

Vancouver

Westwood SJ. The potential existential threat of large language models to online survey research. Proceedings of the National Academy of Sciences. 2025;122(47). doi:10.1073/pnas.2518075122

BibTeX
@article{Westwood2025,
  author = {Westwood, Sean J},
  title = {The potential existential threat of large language models to online survey research},
  journal = {Proceedings of the National Academy of Sciences},
  year = {2025},
  volume = {122},
  number = {47},
  doi = {10.1073/pnas.2518075122}
}
Kostenlos starten →