EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

On Benchmarking Human-like Intelligence in Machines

Lance Ying, Katherine M Collins iD, Lionel Wong, Ilia Sucholutsky iD, Ryan Liu, Adrian Weller iD, Tianmin Shu iD, Thomas L Griffiths iD, Joshua B Tenenbaum iD

Open Mind · 21.08.2026 · 10 · 1394-1412

DOI: 10.1162/opmi.a.385 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Open Mind, online erschienen am 21.08.2026. Verfasst von Lance Ying und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, Harvard University, Cambridge, MA, USA. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 102 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schmidt AI 2050, ONR, MIT-IBM Watson AI Lab.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated data, are increasingly posited as approximate models of human cognition. However, we argue that many current evaluation paradigms for AI are insufficient for assessing human-like cognitive capabilities in these models. We identify a set of key shortcomings: a lack of human-validated labels, inadequate representation of human response variability and uncertainty, and reliance on simplified and ecologically invalid tasks. We support our claims by conducting a human evaluation study on nine existing AI benchmarks, suggesting major limitations in task and label designs. To address these limitations, we propose five concrete recommendations for future AI evaluation efforts that will enable more rigorous and meaningful understanding of human-like cognitive capacities in AI models.

Schlagwörter

Stichwörter der Autor:innen

AI benchmarkingcognitive scienceevaluationhuman variabilityhuman-like intelligence

Autor:innen und Institutionen

  • Department of Psychology, Harvard University, Cambridge, MA, USA.
  • Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
  • Department of Engineering, University of Cambridge, Cambridge, UK.
  • Department of Computer Science, New York University, New York, NY, USA.
  • Department of Computer Science, Princeton University, Princeton, NJ, USA.
  • Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA.

ORCID-Profile: Katherine M Collins, Ilia Sucholutsky, Adrian Weller, Tianmin Shu, Thomas L Griffiths, Joshua B Tenenbaum

Förderung

  • Schmidt AI 2050
  • ONR
  • MIT-IBM Watson AI Lab
  • Reid Hoffman and the Siegel Family Foundation
  • King’s College Cambridge
  • Cambridge Trust
  • Turing AI Fellowship
  • Leverhulme Trust via CFI

Zitieren

APA

Ying, L., Collins, K. M., Wong, L., Sucholutsky, I., Liu, R., Weller, A., Shu, T., Griffiths, T. L., & Tenenbaum, J. B. (2026). On Benchmarking Human-like Intelligence in Machines. Open Mind, 10, 1394-1412. https://doi.org/10.1162/opmi.a.385

Vancouver

Ying L, Collins KM, Wong L, Sucholutsky I, Liu R, Weller A, et al. On Benchmarking Human-like Intelligence in Machines. Open Mind. 2026;10:1394-1412. doi:10.1162/opmi.a.385

BibTeX
@article{Ying2026,
  author = {Ying, Lance and Collins, Katherine M and Wong, Lionel and Sucholutsky, Ilia and Liu, Ryan and Weller, Adrian and Shu, Tianmin and Griffiths, Thomas L and Tenenbaum, Joshua B},
  title = {On Benchmarking Human-like Intelligence in Machines},
  journal = {Open Mind},
  year = {2026},
  volume = {10},
  pages = {1394--1412},
  doi = {10.1162/opmi.a.385}
}
Kostenlos starten →