EUVIMEDSkopri EUVIMED
Open access · CC BY 4.0

Do Large Language Models Know What Humans Know?

Sean Trott, Cameron Jones, Tyler Chang, James Michaelov, Benjamin Bergen

Cognitive Science · 2023-07 · 47 (7)

DOI: 10.1111/cogs.13309 ↗

At a glance

Journal article in Cognitive Science, published online on 2023-07-04. Written by Sean Trott and 4 co-authors. Institution: Department of Cognitive Science, University of California San Diego. Major MEDLINE topics: Communication, Theory of Mind. Cited 91 times so far (Crossref); 77 references. The full text is freely available (open access, CC BY 4.0).

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre‐registered analyses, we present a linguistic version of the False Belief Task to both human participants and a large language model, GPT‐3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans nor does it explain the full extent of their behavior—despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible.

Subjects

MeSH terms (MEDLINE)

CommunicationTheory of MindChildHumansDeceptionLanguageChild Development

Author keywords

Belief attributionFalse Belief TaskLanguageLarge language models

Authors and institutions

  • Department of Cognitive Science, University of California San Diego.

Cite

APA

Trott, S., Jones, C., Chang, T., Michaelov, J., & Bergen, B. (2023). Do Large Language Models Know What Humans Know?. Cognitive Science, 47(7). https://doi.org/10.1111/cogs.13309

Vancouver

Trott S, Jones C, Chang T, Michaelov J, Bergen B. Do Large Language Models Know What Humans Know?. Cognitive Science. 2023;47(7). doi:10.1111/cogs.13309

BibTeX
@article{Trott2023,
  author = {Trott, Sean and Jones, Cameron and Chang, Tyler and Michaelov, James and Bergen, Benjamin},
  title = {Do Large Language Models Know What Humans Know?},
  journal = {Cognitive Science},
  year = {2023},
  volume = {47},
  number = {7},
  doi = {10.1111/cogs.13309}
}
Start for free →