EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Why do experts miss AI’s errors? Evidence from a randomized labeling experiment

Sofoklis Goulas iD, Rigissa Megalokonomou iD, Panagiotis Sotirakopoulos iD

PNAS Nexus · 30.05.2026 · 5 (6)

DOI: 10.1093/pnasnexus/pgag146 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PNAS Nexus, online erschienen am 09.06.2026. Verfasst von Sofoklis Goulas und 2 weiteren Autor:innen. Beteiligte Einrichtung: Research Team, foundry10, 4244 University Way NE, #85538, Seattle, WA 98145, USA. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Monash University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As organizations increasingly rely on algorithmic decision aids, human oversight is vital to prevent automated errors from spreading. But what makes experts correct an algorithm or let it stand? In a preregistered randomized experiment, education experts reviewed identical student work paired with an intentionally inaccurate score labeled as either human- or AI-generated. We independently varied whether the score was too harsh or too lenient. The outcome—the grading fairness gap—measures the distance between the expert’s revised mark and the objective rating. Under a harsh recommendation, the gap was 22% larger when the score was labeled as AI-generated; in the lenient case, the fairness gap under AI and human labels was statistically indistinguishable. Mediation analysis reveals that higher perceived ability and responsibility of the algorithm in the harsh scenario explain over half of the effect, while weaker attributions in the lenient case lead to stricter corrections. Thus, deference to AI depends not on automation itself but on the direction of its errors and the credibility it signals—offering design insights for accountable human–AI collaboration.

Schlagwörter

Stichwörter der Autor:innen

AI in educationhuman–AI collaborationoversight of algorithmic systemsrandomized labeling experiment

Autor:innen und Institutionen

  • Research Team, foundry10, 4244 University Way NE, #85538, Seattle, WA 98145, USA.
  • MacMillan Center, Yale University, 115 Prospect Street, New Haven, CT 06511, USA.
  • Department of Economics, University of Crete, Gallos Campus, Rethymno 74 150, Greece.
  • IZA@LISER - Institute of Labor Economics, Schaumburg-Lippe-Strasse 5-9, Bonn 53113, Germany.
  • Department of Economics, Monash University, 27 Sir John Monash Drive, Caulfield East, Melbourne, VIC 3162, Australia.
  • School of Economics, University of Queensland, St Lucia, Brisbane, QLD 4072, Australia.

ORCID-Profile: Sofoklis Goulas, Rigissa Megalokonomou, Panagiotis Sotirakopoulos

Förderung

  • Monash University

Zitieren

APA

Goulas, S., Megalokonomou, R., & Sotirakopoulos, P. (2026). Why do experts miss AI’s errors? Evidence from a randomized labeling experiment. PNAS Nexus, 5(6). https://doi.org/10.1093/pnasnexus/pgag146

Vancouver

Goulas S, Megalokonomou R, Sotirakopoulos P. Why do experts miss AI’s errors? Evidence from a randomized labeling experiment. PNAS Nexus. 2026;5(6). doi:10.1093/pnasnexus/pgag146

BibTeX
@article{Goulas2026,
  author = {Goulas, Sofoklis and Megalokonomou, Rigissa and Sotirakopoulos, Panagiotis},
  title = {Why do experts miss AI’s errors? Evidence from a randomized labeling experiment},
  journal = {PNAS Nexus},
  year = {2026},
  volume = {5},
  number = {6},
  doi = {10.1093/pnasnexus/pgag146}
}
Kostenlos starten →