EUVIMEDObjevte EUVIMED
Open access · CC BY 4.0

Word Predictability as a Measure of Second Language Proficiency

Langdon Holmes, Scott Crossley, Wesley Morris, Joon Suh Choi

Language Learning · 2026-07-09

DOI: 10.1111/lang.70037 ↗

At a glance

Journal article in Language Learning, published online on 2026-07-09. Written by Langdon Holmes and 3 co-authors. 99 references. The full text is freely available (open access, CC BY 4.0).

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

This study introduces predictability BERT , a novel metric for assessing second language (L2) proficiency based on the predictability of word choices in learner language production. Using BERT (Devlin et al., 2019), we calculated the conditional probability of each word in a text given its surrounding context. We evaluated predictability BERT on two datasets: the Lexical Proficiency Corpus ( N = 480), containing analytic ratings of lexical proficiency, and TOEFL 11 ( N = 11,000; Blanchard et al., 2013), containing standardized language proficiency scores. Results show that predictability BERT correlated with ratings of collocation accuracy ( r = .80) and lexical proficiency ( r = .73). In a multilevel model, predictability BERT was the strongest predictor of language proficiency compared to conventional measures of lexical and phraseological sophistication, explaining 59% of variance in TOEFL scores. These findings suggest high‐proficiency L2 learners make more predictable word choices, supporting usage‐based theories emphasizing the role of statistical learning in L2 development.

Cite

APA

Holmes, L., Crossley, S., Morris, W., & Choi, J. S. (2026). Word Predictability as a Measure of Second Language Proficiency. Language Learning. https://doi.org/10.1111/lang.70037

Vancouver

Holmes L, Crossley S, Morris W, Choi JS. Word Predictability as a Measure of Second Language Proficiency. Language Learning. 2026. doi:10.1111/lang.70037

BibTeX
@article{Holmes2026,
  author = {Holmes, Langdon and Crossley, Scott and Morris, Wesley and Choi, Joon Suh},
  title = {Word Predictability as a Measure of Second Language Proficiency},
  journal = {Language Learning},
  year = {2026},
  doi = {10.1111/lang.70037}
}
Start for free →