Model-Contingent Polarity Bias in Large Language Model Annotation: Implications for Semantic Multimedia Personalization
Computers · 2026-04-22 · 15 (5) · 262
DOI: 10.3390/computers15050262 ↗At a glance
Journal article in Computers, published online on 2026-04-22. Written by Constantinos Djouvas and 3 co-authors. 41 references. The full text is freely available (open access, CC BY 4.0).
Abstract (original language)
Large Language Models (LLMs) are increasingly deployed as automated annotators in semantic multimedia systems, yet their reliability varies significantly across architectures. This study extends prior cross-model evaluations by benchmarking ChatGPT-5, Qwen-3, and Gemini-3-flash against human expert annotations using the HRAST hotel review dataset. We adopt a bias-by-design framework to analyze systematic divergences in sentiment, topic, and aspect labeling across real and synthetic data, while investigating the moderating effects of annotation mode. Findings reveal model-contingent polarity bias: ChatGPT-5 exhibits a pronounced neutrality bias, while Qwen-3 and Gemini-3-flash align more closely with human polarization. Agreement is substantial for concrete topics but diverges on abstract evaluative dimensions. Synthetic data consistently inflates reliability metrics while masking ambiguity. These findings highlight that annotation bias is structurally embedded in model design choices and operational conditions. Cross-architectural triangulation and mode-aware deployment strategies are recommended for robust semantic multimedia system development.
Cite
APA
Djouvas, C., Andreou, C., Voutsa, M. C., & Tsapatsoulis, N. (2026). Model-Contingent Polarity Bias in Large Language Model Annotation: Implications for Semantic Multimedia Personalization. Computers, 15(5), 262. https://doi.org/10.3390/computers15050262
Vancouver
Djouvas C, Andreou C, Voutsa MC, Tsapatsoulis N. Model-Contingent Polarity Bias in Large Language Model Annotation: Implications for Semantic Multimedia Personalization. Computers. 2026;15(5):262. doi:10.3390/computers15050262
BibTeX
@article{Djouvas2026,
author = {Djouvas, Constantinos and Andreou, Christiana and Voutsa, Maria C. and Tsapatsoulis, Nicolas},
title = {Model-Contingent Polarity Bias in Large Language Model Annotation: Implications for Semantic Multimedia Personalization},
journal = {Computers},
year = {2026},
volume = {15},
number = {5},
pages = {262},
doi = {10.3390/computers15050262}
}
EUVIMED