MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages
ACM Transactions on Asian and Low-Resource Language Information Processing · 31.07.2026 · 25 (7) · 1-14
DOI: 10.1145/3812651 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Asian and Low-Resource Language Information Processing, online erschienen am 09.07.2026. Verfasst von Weerayut Buaphet und 10 weiteren Autor:innen. Die Arbeit nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Social media data has been of interest to Natural Language Processing (NLP) practitioners for over a decade, because of its richness in information, but also challenges for automatic processing. Since language use is more informal, spontaneous, and adheres to many different sociolects, the performance of NLP models often deteriorates. One solution to this problem is to transform data to a standard variant before processing it, which is also called lexical normalization. There has been a wide variety of benchmarks and models proposed for this task. The MultiLexNorm benchmark proposed to unify these efforts, but it consists almost solely of languages from the Indo-European language family in the Latin script. Hence, we propose an extension to MultiLexNorm, which covers five Asian languages from different language families in four different scripts. We show that the previous state-of-the-art model performs worse on the new languages and propose a new architecture based on Large Language Models (LLMs), which shows more robust performance. Finally, we analyze remaining errors, revealing future directions for this task.
Zitieren
APA
Buaphet, W., Nguyen, T., Kondo, R., Kajiwara, T., Kim, Y., Lee, J., Lee, H., Lovenia, H., Limkonchotiwat, P., Nutanong, S., & Goot, R. V. D. (2026). MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages. ACM Transactions on Asian and Low-Resource Language Information Processing, 25(7), 1-14. https://doi.org/10.1145/3812651
Vancouver
Buaphet W, Nguyen T, Kondo R, Kajiwara T, Kim Y, Lee J, et al. MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages. ACM Transactions on Asian and Low-Resource Language Information Processing. 2026;25(7):1-14. doi:10.1145/3812651
BibTeX
@article{Buaphet2026,
author = {Buaphet, Weerayut and Nguyen, Thanh-Nhi and Kondo, Risa and Kajiwara, Tomoyuki and Kim, Yumin and Lee, Jimin and Lee, Hwanhee and Lovenia, Holy and Limkonchotiwat, Peerat and Nutanong, Sarana and Goot, Rob Van der},
title = {MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages},
journal = {ACM Transactions on Asian and Low-Resource Language Information Processing},
year = {2026},
volume = {25},
number = {7},
pages = {1--14},
doi = {10.1145/3812651}
}
EUVIMED