EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Automatic register identification for the open web using multilingual deep learning

Erik Henriksson, Amanda Myntti, Saara Hellström, Anni Eskelinen, Selcen Erten-Johansson, Veronika Laippala

Natural Language Processing · 20.11.2026 · 1-50

DOI: 10.1017/nlp.2026.10044 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Natural Language Processing, online erschienen am 20.11.2026. Verfasst von Erik Henriksson und 5 weiteren Autor:innen. Die Arbeit nennt 108 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article presents multilingual deep learning models for identifying web registers—text varieties such as news reports and discussion forums—across 16 languages. We introduce the Multilingual CORE corpora, which contain over 72,000 documents annotated with a hierarchical taxonomy of 25 registers designed to cover the entire open web. Using multi-label classification, our best model achieves 79% F1 averaged across languages, matching or exceeding previous studies that used simpler classification schemes. This demonstrates that models can perform well even with a complex register scheme at multilingual scale. However, we observe a consistent performance ceiling across all models and configurations. When we remove documents with uncertain labels through data pruning, performance increases to over 90% F1, suggesting that this ceiling stems from inherent ambiguity in web registers rather than model limitations. Analysis of hybrid texts (those combining multiple registers) reveals that the main challenge lies not in classifying hybrids themselves, but in distinguishing hybrid from non-hybrid documents. Multilingual models consistently outperform monolingual ones, particularly for languages with limited training data. Zero-shot performance on unseen languages drops by an average of 7%, though this varies by language (3–8%), indicating that while registers share features across languages, they also retain language-specific characteristics.

Zitieren

APA

Henriksson, E., Myntti, A., Hellström, S., Eskelinen, A., Erten-Johansson, S., & Laippala, V. (2026). Automatic register identification for the open web using multilingual deep learning. Natural Language Processing, 1-50. https://doi.org/10.1017/nlp.2026.10044

Vancouver

Henriksson E, Myntti A, Hellström S, Eskelinen A, Erten-Johansson S, Laippala V. Automatic register identification for the open web using multilingual deep learning. Natural Language Processing. 2026:1-50. doi:10.1017/nlp.2026.10044

BibTeX
@article{Henriksson2026,
  author = {Henriksson, Erik and Myntti, Amanda and Hellström, Saara and Eskelinen, Anni and Erten-Johansson, Selcen and Laippala, Veronika},
  title = {Automatic register identification for the open web using multilingual deep learning},
  journal = {Natural Language Processing},
  year = {2026},
  pages = {1--50},
  doi = {10.1017/nlp.2026.10044}
}
Kostenlos starten →