Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research
Sociological Methods & Research · 2025-08 · 54 (3) · 1110-1155
DOI: 10.1177/00491241251342008 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sociological Methods & Research, online erschienen am 21.05.2025. Verfasst von Alex Lyman und 5 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 92 Literaturangaben. Gefördert von: Brigham Young University.
Abstract (Originalsprache)
Generative artificial intelligence (AI) has the potential to revolutionize social science research. However, researchers face the difficult challenge of choosing a specific AI model, often without social science-specific guidance. To demonstrate the importance of this choice, we present an evaluation of the effect of alignment, or human-driven modification, on the ability of large language models (LLMs) to simulate the attitudes of human populations (sometimes called silicon sampling ). We benchmark aligned and unaligned versions of six open-source LLMs against each other and compare them to similar responses by humans. Our results suggest that model alignment impacts output in predictable ways, with implications for prompting, task completion, and the substantive content of LLM-based results. We conclude that researchers must be aware of the complex ways in which model training affects their research and carefully consider model choice for each project. We discuss future steps to improve how social scientists work with generative AI tools.
Förderung
- Brigham Young University
Zitieren
APA
Lyman, A., Hepner, B., Argyle, L. P., Busby, E. C., Gubler, J. R., & Wingate, D. (2025). Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research. Sociological Methods & Research, 54(3), 1110-1155. https://doi.org/10.1177/00491241251342008
Vancouver
Lyman A, Hepner B, Argyle LP, Busby EC, Gubler JR, Wingate D. Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research. Sociological Methods & Research. 2025;54(3):1110-1155. doi:10.1177/00491241251342008
BibTeX
@article{Lyman2025,
author = {Lyman, Alex and Hepner, Bryce and Argyle, Lisa P. and Busby, Ethan C. and Gubler, Joshua R. and Wingate, David},
title = {Balancing Large Language Model Alignment and Algorithmic Fidelity in Social Science Research},
journal = {Sociological Methods & Research},
year = {2025},
volume = {54},
number = {3},
pages = {1110--1155},
doi = {10.1177/00491241251342008}
}
EUVIMED