EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Human-AI rating asymmetries across evaluative dimensions in instruction-tuned open-weight language models: A descriptive multi-model audit

Wenjun Tian

PLOS One · 17.11.2026 · 21 (11) · e0354555

DOI: 10.1371/journal.pone.0354555 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 17.11.2026. Verfasst von Wenjun Tian. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Language models increasingly evaluate people and automated systems, but how they rate a human relative to an artificial intelligence (AI) system on the same evaluative attribute has received little systematic description. We audited seven instruction-tuned open-weight language models from six families. In June 2026, each model rated “a human” and “an AI system” in separate prompts on a 1–7 scale for eight attributes (creative, reliable, productive, good at making decisions, morally sound, emotionally perceptive, trustworthy, intrinsically valuable), in English and Chinese, with four wordings and five or eight repetitions (6,784 responses). Under a 12-token output limit, 13.7% of responses gave no usable rating. Overall, the human was rated about half a scale point higher, but the difference depended on the attribute. Human-favoured ratings of intrinsic worth and creativity remained significant across the principal sensitivity analyses; AI-favoured productivity and human-favoured emotional-perceptiveness differences depended on prompt language and level of inference; a moral-judgment difference was small and not robust; and no direction was established for decision-making, reliability or trustworthiness. The pre-specified expectation that all competence attributes favour the AI system did not hold. In September 2026, the same checkpoints served in a different environment reproduced the profile (correlation of dimension means 0.98), and no systematic change was detected with a larger output limit. When six referents were rated together on one scale, moral judgment and trustworthiness became human-favoured and the creativity difference decreased in English prompts, to a value no longer distinguishable from zero; in isolated Chinese prompts, it decreased when “a person” replaced “humans”. The ratings are reproducible for these checkpoints and this design but depend on the rating format and the wording of the human referent. Because all models were instruction-tuned, the audit cannot separate patterns acquired in pretraining from those added by instruction and safety training.

Zitieren

APA

Tian, W. (2026). Human-AI rating asymmetries across evaluative dimensions in instruction-tuned open-weight language models: A descriptive multi-model audit. PLOS One, 21(11), e0354555. https://doi.org/10.1371/journal.pone.0354555

Vancouver

Tian W. Human-AI rating asymmetries across evaluative dimensions in instruction-tuned open-weight language models: A descriptive multi-model audit. PLOS One. 2026;21(11):e0354555. doi:10.1371/journal.pone.0354555

BibTeX
@article{Tian2026,
  author = {Tian, Wenjun},
  title = {Human-AI rating asymmetries across evaluative dimensions in instruction-tuned open-weight language models: A descriptive multi-model audit},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {11},
  pages = {e0354555},
  doi = {10.1371/journal.pone.0354555}
}
Kostenlos starten →