EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Community size rather than grammatical complexity better predicts Large Language Model accuracy in a novel Wug Test

Nikoleta Pantelidou, Evelina Leivada, Raquel Montero, Paolo Morosi iD

PLOS One · 11.03.2026 · 21 (3) · e0343164

DOI: 10.1371/journal.pone.0343164 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 11.03.2026. Verfasst von Nikoleta Pantelidou und 3 weiteren Autor:innen. Beteiligte Einrichtung: Departament de Filologia Catalana, Centre de Lingüística Teòrica, Universitat Autònoma de Barcelona, Bellaterra, Barcelona, Spain. Hauptthemen laut MEDLINE: Linguistics, Language, Language Tests. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministerio de Ciencia e Innovación.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The linguistic abilities of Large Language Models are a matter of ongoing debate. This study contributes to this discussion by investigating model performance in a morphological generalization task that involves novel words. Using a multilingual adaptation of the Wug Test, six models were tested across four partially unrelated languages (Catalan, English, Greek, and Spanish) and compared with human speakers. The aim is to determine whether model accuracy approximates human competence and whether it is shaped primarily by linguistic complexity or by the size of the linguistic community, which affects the quantity of available training data. Consistent with previous research, the results show that the models are able to generalize morphological processes to unseen words with human-like accuracy. However, accuracy patterns align more closely with community size and data availability than with structural complexity, refining earlier claims in the literature. In particular, languages with larger speaker communities and stronger digital representation, such as Spanish and English, revealed higher accuracy than less-resourced ones like Catalan and Greek. Overall, our findings suggest that model behavior is mainly driven by the richness of linguistic resources rather than by sensitivity to grammatical complexity, reflecting a form of performance that resembles human linguistic competence only superficially.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LinguisticsLanguageLanguage TestsHumansLarge Language ModelsMultilingualism

Autor:innen und Institutionen

  • Departament de Filologia Catalana, Centre de Lingüística Teòrica, Universitat Autònoma de Barcelona, Bellaterra, Barcelona, Spain.
  • Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona, Spain.

ORCID-Profile: Paolo Morosi

Förderung

  • Ministerio de Ciencia e Innovación

Zitieren

APA

Pantelidou, N., Leivada, E., Montero, R., & Morosi, P. (2026). Community size rather than grammatical complexity better predicts Large Language Model accuracy in a novel Wug Test. PLOS One, 21(3), e0343164. https://doi.org/10.1371/journal.pone.0343164

Vancouver

Pantelidou N, Leivada E, Montero R, Morosi P. Community size rather than grammatical complexity better predicts Large Language Model accuracy in a novel Wug Test. PLOS One. 2026;21(3):e0343164. doi:10.1371/journal.pone.0343164

BibTeX
@article{Pantelidou2026,
  author = {Pantelidou, Nikoleta and Leivada, Evelina and Montero, Raquel and Morosi, Paolo},
  title = {Community size rather than grammatical complexity better predicts Large Language Model accuracy in a novel Wug Test},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {3},
  pages = {e0343164},
  doi = {10.1371/journal.pone.0343164}
}
Kostenlos starten →