A moral Turing test: How belief and source shape detection of and agreement with LLM judgments
PLOS One · 05.08.2026 · 21 (8) · e0353391
DOI: 10.1371/journal.pone.0353391 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 05.08.2026. Verfasst von Basile Garcia und 2 weiteren Autor:innen. Beteiligte Einrichtung: University of Geneva, Geneva, Switzerland. Hauptthemen laut MEDLINE: Large Language Models, Judgment, Morals. Die Arbeit nennt 73 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: H2020 European Research Council, Agence Nationale de la Recherche, Alexander von Humboldt-Stiftung.
Abstract (Originalsprache)
As large language models (LLMs) are increasingly integrated into decision-making systems (e.g., autonomous vehicles and medical devices), understanding how humans perceive and evaluate AI-generated judgments is crucial. To investigate this, we conducted a series of experiments in which participants evaluated justifications for moral and non-moral choices, generated either by humans or LLMs. Participants attempted to identify the source of each justification (either human or LLM) and indicated their agreement with its content. We found that while detection accuracy was consistently above chance, it remained below 75%. In terms of agreement, there was no overall preference for human-generated responses, even though machine-generated justifications were favored in particularly challenging moral scenarios. Notably, we observed a systematic anti-AI bias: participants were less likely to agree with judgments they believed were AI-generated, regardless of the true source. Linguistic cues, such as response length, typos, first-person pronouns, and cost-benefit language markers (e.g., “lives,” “save”), influenced both detection and agreement. Participants tended to disagree with cost-benefit calculations, possibly due to an expectation that AI would favor such reasoning. These findings highlight the influence of motivated belief and ingroup/outgroup bias in shaping human evaluation of AI-generated content, particularly in morally sensitive contexts.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- University of Geneva, Geneva, Switzerland.
- Google DeepMind, New York, New York, United States of America.
- Département d'études cognitives, École normale supérieure, PSL Research University, Paris, France.
- Laboratoire de Neurosciences Cognitives Computationnelles, Institut National de la Santé et de la Recherche Médicale, Paris, France.
ORCID-Profile: Basile Garcia, Stefano Palminteri
Förderung
- H2020 European Research Council
- Agence Nationale de la Recherche
- Alexander von Humboldt-Stiftung
Zitieren
APA
Garcia, B., Qian, C., & Palminteri, S. (2026). A moral Turing test: How belief and source shape detection of and agreement with LLM judgments. PLOS One, 21(8), e0353391. https://doi.org/10.1371/journal.pone.0353391
Vancouver
Garcia B, Qian C, Palminteri S. A moral Turing test: How belief and source shape detection of and agreement with LLM judgments. PLOS One. 2026;21(8):e0353391. doi:10.1371/journal.pone.0353391
BibTeX
@article{Garcia2026,
author = {Garcia, Basile and Qian, Crystal and Palminteri, Stefano},
title = {A moral Turing test: How belief and source shape detection of and agreement with LLM judgments},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {8},
pages = {e0353391},
doi = {10.1371/journal.pone.0353391}
}
EUVIMED