EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating the language abilities of Large Language Models vs. humans: Three caveats

Evelina Leivada, Vittoria Dentella, Fritz Günther

Biolinguistics · 19.04.2024 · 18

DOI: 10.5964/bioling.14391 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biolinguistics, online erschienen am 19.04.2024. Verfasst von Evelina Leivada und 2 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 14 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We identify and analyze three caveats that may arise when analyzing the linguistic abilities of Large Language Models. The problem of unlicensed generalizations refers to the danger of interpreting performance in one task as predictive of the models’ overall capabilities, based on the assumption that because a specific task performance is indicative of certain underlying capabilities in humans, the same association holds for models. The human-like paradox refers to the problem of lacking human comparisons, while at the same time attributing human-like abilities to the models. Last, the problem of double standards refers to the use of tasks and methodologies that either cannot be applied to humans or they are evaluated differently in models vs. humans. While we recognize the impressive linguistic abilities of LLMs, we conclude that specific claims about the models’ human-likeness in the grammatical domain are premature.

Zitieren

APA

Leivada, E., Dentella, V., & Günther, F. (2024). Evaluating the language abilities of Large Language Models vs. humans: Three caveats. Biolinguistics, 18. https://doi.org/10.5964/bioling.14391

Vancouver

Leivada E, Dentella V, Günther F. Evaluating the language abilities of Large Language Models vs. humans: Three caveats. Biolinguistics. 2024;18. doi:10.5964/bioling.14391

BibTeX
@article{Leivada2024,
  author = {Leivada, Evelina and Dentella, Vittoria and Günther, Fritz},
  title = {Evaluating the language abilities of Large Language Models vs. humans: Three caveats},
  journal = {Biolinguistics},
  year = {2024},
  volume = {18},
  doi = {10.5964/bioling.14391}
}
Kostenlos starten →