A corpus-based approach to identifying stopwords in multiple languages
Journal of the Digital Humanities Association of Southern Africa (DHASA) · 20.11.2026 · 7 (2) · 56-66
DOI: 10.55492/dhasa.v7i02.7691 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the Digital Humanities Association of Southern Africa (DHASA), online erschienen am 20.11.2026. Verfasst von Bronwyn Bowles-King und 6 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Stopwords are generally considered common, high-frequency words that carry little semantic information, and they are often filtered out during preprocessing, for example, to save computational power, improve model performance, or support focused analysis. However, not all languages have a stopword list readily available. This study identifies statistical word features in general-purpose corpora to evaluate existing Afrikaans, English, isiZulu, and Sesotho stopword lists. We consider whether stopword features differ for these languages compared to features typically used to identify English stopwords, such as word length, word frequency, document frequency, and combinations of these features. Although stopwords are ubiquitous in the corpora, they are a tiny minority group in terms of the number of unique words. This skewed distribution raises challenges in statistical modelling as it biases the classifier towards the larger group of content words. We therefore applied elastic net logistic regression with feature ablation for 25 features. The most effective class differentiation was achieved using a reduced trivariate model including word length, log term frequency, and log document frequency, with AUCs between 98.20 and 99.99 for the four languages. Out-of-fold evaluations for this core model demonstrated low precision (0.87--6.14) and F1-scores (1.73--11.53), but high recall (91.36--100.00). Our findings suggest that these three statistical measures for stopword identification behave consistently across the target languages, regardless of orthography or corpus size, and they display the potential to filter candidate stopwords for subsequent evaluation and curation.
Zitieren
APA
Bowles-King, B., Gaustad, T., Matfunjwa, M., Skosana, N., Trollip, B., White, M., & Zaanen, M. V. (2026). A corpus-based approach to identifying stopwords in multiple languages. Journal of the Digital Humanities Association of Southern Africa (DHASA), 7(2), 56-66. https://doi.org/10.55492/dhasa.v7i02.7691
Vancouver
Bowles-King B, Gaustad T, Matfunjwa M, Skosana N, Trollip B, White M, et al. A corpus-based approach to identifying stopwords in multiple languages. Journal of the Digital Humanities Association of Southern Africa (DHASA). 2026;7(2):56-66. doi:10.55492/dhasa.v7i02.7691
BibTeX
@article{BowlesKing2026,
author = {Bowles-King, Bronwyn and Gaustad, Tanja and Matfunjwa, Muzi and Skosana, Nomsa and Trollip, Benito and White, Michelle and Zaanen, Menno Van},
title = {A corpus-based approach to identifying stopwords in multiple languages},
journal = {Journal of the Digital Humanities Association of Southern Africa (DHASA)},
year = {2026},
volume = {7},
number = {2},
pages = {56--66},
doi = {10.55492/dhasa.v7i02.7691}
}
EUVIMED