EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Large language models are able to downplay their cognitive abilities to fit the persona they simulate

Jiří Milička, Anna Marklová iD, Klára VanSlambrouck, Eva Pospíšilová, Jana Šimsová, Samuel Harvan, Ondřej Drobil

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0298522 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Jiří Milička und 6 weiteren Autor:innen. Beteiligte Einrichtung: Institute of the Czech National Corpus, Faculty of Arts, Charles University, Prague, Czech Republic. Hauptthemen laut MEDLINE: Cognition, Language. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Czech Science Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study explores the capabilities of large language models to replicate the behavior of individuals with underdeveloped cognitive and language skills. Specifically, we investigate whether these models can simulate child-like language and cognitive development while solving false-belief tasks, namely, change-of-location and unexpected-content tasks. GPT-3.5-turbo and GPT-4 models by OpenAI were prompted to simulate children (N = 1296) aged one to six years. This simulation was instantiated through three types of prompts: plain zero-shot, chain-of-thoughts, and primed-by-corpus. We evaluated the correctness of responses to assess the models’ capacity to mimic the cognitive skills of the simulated children. Both models displayed a pattern of increasing correctness in their responses and rising language complexity. That is in correspondence with a gradual enhancement in linguistic and cognitive abilities during child development, which is described in the vast body of research literature on child development. GPT-4 generally exhibited a closer alignment with the developmental curve observed in ‘real’ children. However, it displayed hyper-accuracy under certain conditions, notably in the primed-by-corpus prompt type. Task type, prompt type, and the choice of language model influenced developmental patterns, while temperature and the gender of the simulated parent and child did not consistently impact results. We conducted analyses of linguistic complexity, examining utterance length and Kolmogorov complexity. These analyses revealed a gradual increase in linguistic complexity corresponding to the age of the simulated children, regardless of other variables. These findings show that the language models are capable of downplaying their abilities to achieve a faithful simulation of prompted personas.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

CognitionLanguageHumansChild DevelopmentLinguisticsAptitude

Autor:innen und Institutionen

  • Institute of the Czech National Corpus, Faculty of Arts, Charles University, Prague, Czech Republic.
  • Department of Slavic and Hungarian Studies, Faculty of Language, Literature and Humanities, Humboldt University of Berlin, Berlin, Germany.
  • Faculty of Arts, Charles University, Prague, Czech Republic.

ORCID-Profile: Anna Marklová

Förderung

  • Czech Science Foundation

Zitieren

APA

Milička, J., Marklová, A., VanSlambrouck, K., Pospíšilová, E., Šimsová, J., Harvan, S., & Drobil, O. (2024). Large language models are able to downplay their cognitive abilities to fit the persona they simulate. PLOS ONE. https://doi.org/10.1371/journal.pone.0298522

Vancouver

Milička J, Marklová A, VanSlambrouck K, Pospíšilová E, Šimsová J, Harvan S, et al. Large language models are able to downplay their cognitive abilities to fit the persona they simulate. PLOS ONE. 2024. doi:10.1371/journal.pone.0298522

BibTeX
@article{Milika2024,
  author = {Milička, Jiří and Marklová, Anna and VanSlambrouck, Klára and Pospíšilová, Eva and Šimsová, Jana and Harvan, Samuel and Drobil, Ondřej},
  title = {Large language models are able to downplay their cognitive abilities to fit the persona they simulate},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0298522}
}
Kostenlos starten →