Too many tokens: Modeling lexical effects in corpora
Glossa Psycholinguistics · 10.08.2026 · 5 (1)
DOI: 10.5070/g6011.49062 ↗Kurz gefasst
Fachartikel in der Zeitschrift Glossa Psycholinguistics, online erschienen am 10.08.2026. Verfasst von Susanne Gahl und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Many statistical models of properties of words in texts and in running speech use as their outcome variables properties of word tokens, rather than properties aggregated over word types. Here, we describe methodological and conceptual problems with token-level (or token-based) models, including assumption violations of the models typically used, model redundancy, and distortions of parameters estimating lexical effects – i.e. the very target of what such models seek to capture. We show that these problems persist – and in some ways, grow more extreme – when we follow widely-used practices for specifying random effects structure. We do so by comparing models of spoken word duration of homophones in the Switchboard corpus, a dataset that has been discussed and modeled extensively. We discuss possible remedies and recommendations.
Zitieren
APA
Gahl, S., & Baayen, H. (2026). Too many tokens: Modeling lexical effects in corpora. Glossa Psycholinguistics, 5(1). https://doi.org/10.5070/g6011.49062
Vancouver
Gahl S, Baayen H. Too many tokens: Modeling lexical effects in corpora. Glossa Psycholinguistics. 2026;5(1). doi:10.5070/g6011.49062
BibTeX
@article{Gahl2026,
author = {Gahl, Susanne and Baayen, Harald},
title = {Too many tokens: Modeling lexical effects in corpora},
journal = {Glossa Psycholinguistics},
year = {2026},
volume = {5},
number = {1},
doi = {10.5070/g6011.49062}
}
EUVIMED