EUVIMEDEUVIMED entdecken
Crossref

Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research

Alex Lyman, Bryce Hepner, Lisa P. Argyle, Ethan C. Busby, Joshua R. Gubler, David Wingate

Sociological Methods & Research · 2025-08 · 54 (3) · 1110-1155

DOI: 10.1177/00491241251342008 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sociological Methods & Research, online erschienen am 21.05.2025. Verfasst von Alex Lyman und 5 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 92 Literaturangaben. Gefördert von: Brigham Young University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Generative artificial intelligence (AI) has the potential to revolutionize social science research. However, researchers face the difficult challenge of choosing a specific AI model, often without social science-specific guidance. To demonstrate the importance of this choice, we present an evaluation of the effect of alignment, or human-driven modification, on the ability of large language models (LLMs) to simulate the attitudes of human populations (sometimes called silicon sampling ). We benchmark aligned and unaligned versions of six open-source LLMs against each other and compare them to similar responses by humans. Our results suggest that model alignment impacts output in predictable ways, with implications for prompting, task completion, and the substantive content of LLM-based results. We conclude that researchers must be aware of the complex ways in which model training affects their research and carefully consider model choice for each project. We discuss future steps to improve how social scientists work with generative AI tools.

Förderung

  • Brigham Young University

Zitieren

APA

Lyman, A., Hepner, B., Argyle, L. P., Busby, E. C., Gubler, J. R., & Wingate, D. (2025). Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research. Sociological Methods & Research, 54(3), 1110-1155. https://doi.org/10.1177/00491241251342008

Vancouver

Lyman A, Hepner B, Argyle LP, Busby EC, Gubler JR, Wingate D. Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research. Sociological Methods & Research. 2025;54(3):1110-1155. doi:10.1177/00491241251342008

BibTeX
@article{Lyman2025,
  author = {Lyman, Alex and Hepner, Bryce and Argyle, Lisa P. and Busby, Ethan C. and Gubler, Joshua R. and Wingate, David},
  title = {Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research},
  journal = {Sociological Methods & Research},
  year = {2025},
  volume = {54},
  number = {3},
  pages = {1110--1155},
  doi = {10.1177/00491241251342008}
}
Kostenlos starten →