EUVIMEDEUVIMED entdecken
Crossref

Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research

Hao Lin, Yongjun Zhang

Social Science Computer Review · 2026-06 · 44 (3) · 403-427

DOI: 10.1177/08944393251366243 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Social Science Computer Review, online erschienen am 08.08.2025. Verfasst von Hao Lin und 1 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 61 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have the potential to revolutionize computational social science, particularly in automated textual analysis. In this paper, we conduct a systematic evaluation of the promises and risks associated with using LLMs for text classification tasks, using social movement studies as an example. We propose a framework for social scientists to incorporate LLMs into text annotation, either as the primary coding decision-maker or as a coding assistant. This framework offers researchers tools to develop the potential best-performing prompt, and to systematically examine and report the validity and reliability of LLMs as a methodological tool. Additionally, we evaluate and discuss its epistemic risks associated with validity, reliability, replicability, and transparency. We conclude with several practical guidelines for using LLMs in text annotation tasks and offer recommendations for more effectively communicating epistemic risks in research.

Zitieren

APA

Lin, H., & Zhang, Y. (2026). Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research. Social Science Computer Review, 44(3), 403-427. https://doi.org/10.1177/08944393251366243

Vancouver

Lin H, Zhang Y. Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research. Social Science Computer Review. 2026;44(3):403-427. doi:10.1177/08944393251366243

BibTeX
@article{Lin2026,
  author = {Lin, Hao and Zhang, Yongjun},
  title = {Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research},
  journal = {Social Science Computer Review},
  year = {2026},
  volume = {44},
  number = {3},
  pages = {403--427},
  doi = {10.1177/08944393251366243}
}
Kostenlos starten →