EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models

Emily Robinson, Ranadheer Malla, Chico Q. Camargo

Social Science Computer Review · 30.10.2026

DOI: 10.1177/08944393261501889 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Social Science Computer Review, online erschienen am 30.10.2026. Verfasst von Emily Robinson und 2 weiteren Autor:innen. Die Arbeit nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Framing research has long relied on human annotation, but many aspects of framing – especially framing functions such as defining problems, identifying causes, making moral evaluations, or proposing solutions are difficult to label consistently and at scale. We demonstrate that large language models can reliably identify these framing functions, achieving levels of agreement comparable to those of human annotators. We evaluate three state-of-the-art models (GPT-5, Claude 3.7, and DeepSeek) alongside three trained human coders on a new dataset of social media posts about electric-vehicles, a highly topical and contested issue that holds many interpretations. This setting provides a strong test of both agreement and meaningful disagreement in framing-function annotation. Across models and humans, we find not only substantial convergence but also patterns of valid disagreement – differences that reflect legitimate alternative interpretations rather than annotation error. Recognising and preserving these cases enhances construct validity and provides a more realistic portrayal of how framing functions operate in contested debates. Our results demonstrate that large language models can scale framing function analysis while capturing the variability inherent in human interpretation, enabling forms of communication research that were previously impractical with human-only coding.

Zitieren

APA

Robinson, E., Malla, R., & Camargo, C. Q. (2026). Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models. Social Science Computer Review. https://doi.org/10.1177/08944393261501889

Vancouver

Robinson E, Malla R, Camargo CQ. Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models. Social Science Computer Review. 2026. doi:10.1177/08944393261501889

BibTeX
@article{Robinson2026,
  author = {Robinson, Emily and Malla, Ranadheer and Camargo, Chico Q.},
  title = {Agreement Helps, Disagreement Matters: Scaling Framing Function Annotation With Large Language Models},
  journal = {Social Science Computer Review},
  year = {2026},
  doi = {10.1177/08944393261501889}
}
Kostenlos starten →