Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models
Social Science Computer Review · 30.10.2026
DOI: 10.1177/08944393261501889 ↗Kurz gefasst
Fachartikel in der Zeitschrift Social Science Computer Review, online erschienen am 30.10.2026. Verfasst von Emily Robinson und 2 weiteren Autor:innen. Die Arbeit nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Framing research has long relied on human annotation, but many aspects of framing – especially framing functions such as defining problems, identifying causes, making moral evaluations, or proposing solutions are difficult to label consistently and at scale. We demonstrate that large language models can reliably identify these framing functions, achieving levels of agreement comparable to those of human annotators. We evaluate three state-of-the-art models (GPT-5, Claude 3.7, and DeepSeek) alongside three trained human coders on a new dataset of social media posts about electric-vehicles, a highly topical and contested issue that holds many interpretations. This setting provides a strong test of both agreement and meaningful disagreement in framing-function annotation. Across models and humans, we find not only substantial convergence but also patterns of valid disagreement – differences that reflect legitimate alternative interpretations rather than annotation error. Recognising and preserving these cases enhances construct validity and provides a more realistic portrayal of how framing functions operate in contested debates. Our results demonstrate that large language models can scale framing function analysis while capturing the variability inherent in human interpretation, enabling forms of communication research that were previously impractical with human-only coding.
Zitieren
APA
Robinson, E., Malla, R., & Camargo, C. Q. (2026). Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models. Social Science Computer Review. https://doi.org/10.1177/08944393261501889
Vancouver
Robinson E, Malla R, Camargo CQ. Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models. Social Science Computer Review. 2026. doi:10.1177/08944393261501889
BibTeX
@article{Robinson2026,
author = {Robinson, Emily and Malla, Ranadheer and Camargo, Chico Q.},
title = {Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models},
journal = {Social Science Computer Review},
year = {2026},
doi = {10.1177/08944393261501889}
}
EUVIMED