EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models

Abishek Thamma, Micha Heilbron

Transactions of the Association for Computational Linguistics · 01.06.2026 · 14 · 877-892

DOI: 10.1162/tacl.a.688 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 01.06.2026. Verfasst von Abishek Thamma und 1 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human memory is fleeting. As words are processed, the exact wordforms that make up incoming sentences are rapidly lost. Cognitive scientists have long believed that this limitation of working memory may, paradoxically, help in learning language – an idea supported by classic connectionist modelling work. The rise of Transformers appears to challenge this idea, as these models can learn language effectively, despite lacking working memory limitations or other architectural recency biases. Here, we investigate the hypothesized benefit of fleeting memory for language learning in tightly controlled experiments on transformer language models. Training transformers with and without fleeting memory on a developmentally realistic training set, we find that fleeting memory consistently improves language learning (as quantified by both overall language modelling performance and targeted syntactic evaluation) but, unexpectedly, impairs surprisal-based prediction of human reading times. Interestingly, follow up analyses revealed that this discrepancy – better language modeling, yet worse reading time prediction – could not be accounted for by prior explanations of why better language models sometimes fit human reading time worse. Together, these results support a benefit of memory limitations on neural network language learning – but not on predicting behavior.

Zitieren

APA

Thamma, A., & Heilbron, M. (2026). Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models. Transactions of the Association for Computational Linguistics, 14, 877-892. https://doi.org/10.1162/tacl.a.688

Vancouver

Thamma A, Heilbron M. Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models. Transactions of the Association for Computational Linguistics. 2026;14:877-892. doi:10.1162/tacl.a.688

BibTeX
@article{Thamma2026,
  author = {Thamma, Abishek and Heilbron, Micha},
  title = {Human-like Fleeting Memory Improves Language Learning but Impairs Reading Time Prediction in Transformer Language Models},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  volume = {14},
  pages = {877--892},
  doi = {10.1162/tacl.a.688}
}
Kostenlos starten →