EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Eye Tracking–Based Cognitive Evaluation of Automatic Readability Assessment Methods

Keren Gruteke Klein, Shachar Frenkel, Omer Shubi, Yevgeni Berzak

Computational Linguistics · 23.10.2026 · 1-30

DOI: 10.1162/coli.a.649 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computational Linguistics, online erschienen am 23.10.2026. Verfasst von Keren Gruteke Klein und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automatic methods for scoring text readability have been studied for over a century, and are widely used in research and in user-facing applications in many domains. Thus far, the development and evaluation of such methods have primarily relied on two types of offline human behavioral data, performance on reading comprehension tests and ratings of text readability levels. In this work, we instead focus on a fundamental and understudied aspect of readability, real-time reading ease, captured with online reading measures using eye tracking. We introduce a new cognitive evaluation framework for readability scoring methods that quantifies their ability to account for reading ease, while controlling for content variation across texts. Applying this evaluation to prominent traditional readability formulas, NLP-based methods, commercial systems used in education, and frontier LLMs suggests that they are all poor predictors of English reading ease in adults as compared to word properties commonly used in psycholinguistics for the prediction of reading times. This outcome holds across L1 and L2 speakers, different reading regimes, and textual units of different lengths. Our results reveal an important limitation of a wide range of methods for readability scoring, highlight the utility of real-time behavioral benchmarks for readability research, and call for new, cognitively driven readability scoring approaches that can better account for how humans experience texts in real time.

Zitieren

APA

Klein, K. G., Frenkel, S., Shubi, O., & Berzak, Y. (2026). Eye Tracking–Based Cognitive Evaluation of Automatic Readability Assessment Methods. Computational Linguistics, 1-30. https://doi.org/10.1162/coli.a.649

Vancouver

Klein KG, Frenkel S, Shubi O, Berzak Y. Eye Tracking–Based Cognitive Evaluation of Automatic Readability Assessment Methods. Computational Linguistics. 2026:1-30. doi:10.1162/coli.a.649

BibTeX
@article{Klein2026,
  author = {Klein, Keren Gruteke and Frenkel, Shachar and Shubi, Omer and Berzak, Yevgeni},
  title = {Eye Tracking–Based Cognitive Evaluation of Automatic Readability Assessment Methods},
  journal = {Computational Linguistics},
  year = {2026},
  pages = {1--30},
  doi = {10.1162/coli.a.649}
}
Kostenlos starten →