EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Too many tokens: Modeling lexical effects in corpora

Susanne Gahl, Harald Baayen

Glossa Psycholinguistics · 10.08.2026 · 5 (1)

DOI: 10.5070/g6011.49062 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Glossa Psycholinguistics, online erschienen am 10.08.2026. Verfasst von Susanne Gahl und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Many statistical models of properties of words in texts and in running speech use as their outcome variables properties of word tokens, rather than properties aggregated over word types. Here, we describe methodological and conceptual problems with token-level (or token-based) models, including assumption violations of the models typically used, model redundancy, and distortions of parameters estimating lexical effects – i.e. the very target of what such models seek to capture. We show that these problems persist – and in some ways, grow more extreme – when we follow widely-used practices for specifying random effects structure. We do so by comparing models of spoken word duration of homophones in the Switchboard corpus, a dataset that has been discussed and modeled extensively. We discuss possible remedies and recommendations.

Zitieren

APA

Gahl, S., & Baayen, H. (2026). Too many tokens: Modeling lexical effects in corpora. Glossa Psycholinguistics, 5(1). https://doi.org/10.5070/g6011.49062

Vancouver

Gahl S, Baayen H. Too many tokens: Modeling lexical effects in corpora. Glossa Psycholinguistics. 2026;5(1). doi:10.5070/g6011.49062

BibTeX
@article{Gahl2026,
  author = {Gahl, Susanne and Baayen, Harald},
  title = {Too many tokens: Modeling lexical effects in corpora},
  journal = {Glossa Psycholinguistics},
  year = {2026},
  volume = {5},
  number = {1},
  doi = {10.5070/g6011.49062}
}
Kostenlos starten →