EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

SOCbot: Using Large Language Models to Dynamically Measure and Classify Occupations in Surveys

Patrick Sturgis, Thomas S. Robinson, Laura Fung, Caroline Roberts

Sociological Methods & Research · 29.06.2026

DOI: 10.1177/00491241261461516 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sociological Methods & Research, online erschienen am 29.06.2026. Verfasst von Patrick Sturgis und 3 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Economic and Social Research Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present the results of a new approach to measuring the occupations of respondents in surveys using Large Language Models (LLMs). In our new approach, which we call SOCbot, an LLM integrated in the questionnaire scripting software is used to code the job title to the occupational classification in real-time during the interview. Where the job title does not contain sufficient information to be coded with confidence, the LLM probes for further relevant details on job tasks, industry, qualifications, and so on. SOCbot can also be used offline on already collected response data. Our results demonstrate that the approach attains rates of coder reliability comparable to trained human coders, with consistent performance across four major commercial and open-weight model families. SOCbot can also be deployed using publicly available open-weight models with only a small but measurable accuracy penalty, allowing even users with stringent data-protection constraints to use it. We also demonstrate that the approach is feasible in large-scale survey operations and has significant potential to reduce respondent burden, lower costs, and yield more timely and accurate data.

Förderung

  • Economic and Social Research Council

Zitieren

APA

Sturgis, P., Robinson, T. S., Fung, L., & Roberts, C. (2026). SOCbot: Using Large Language Models to Dynamically Measure and Classify Occupations in Surveys. Sociological Methods & Research. https://doi.org/10.1177/00491241261461516

Vancouver

Sturgis P, Robinson TS, Fung L, Roberts C. SOCbot: Using Large Language Models to Dynamically Measure and Classify Occupations in Surveys. Sociological Methods & Research. 2026. doi:10.1177/00491241261461516

BibTeX
@article{Sturgis2026,
  author = {Sturgis, Patrick and Robinson, Thomas S. and Fung, Laura and Roberts, Caroline},
  title = {SOCbot: Using Large Language Models to Dynamically Measure and Classify Occupations in Surveys},
  journal = {Sociological Methods & Research},
  year = {2026},
  doi = {10.1177/00491241261461516}
}
Kostenlos starten →