EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages

Weerayut Buaphet, Thanh-Nhi Nguyen, Risa Kondo, Tomoyuki Kajiwara, Yumin Kim, Jimin Lee, Hwanhee Lee, Holy Lovenia, Peerat Limkonchotiwat, Sarana Nutanong, Rob Van der Goot

ACM Transactions on Asian and Low-Resource Language Information Processing · 31.07.2026 · 25 (7) · 1-14

DOI: 10.1145/3812651 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Asian and Low-Resource Language Information Processing, online erschienen am 09.07.2026. Verfasst von Weerayut Buaphet und 10 weiteren Autor:innen. Die Arbeit nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Social media data has been of interest to Natural Language Processing (NLP) practitioners for over a decade, because of its richness in information, but also challenges for automatic processing. Since language use is more informal, spontaneous, and adheres to many different sociolects, the performance of NLP models often deteriorates. One solution to this problem is to transform data to a standard variant before processing it, which is also called lexical normalization. There has been a wide variety of benchmarks and models proposed for this task. The MultiLexNorm benchmark proposed to unify these efforts, but it consists almost solely of languages from the Indo-European language family in the Latin script. Hence, we propose an extension to MultiLexNorm, which covers five Asian languages from different language families in four different scripts. We show that the previous state-of-the-art model performs worse on the new languages and propose a new architecture based on Large Language Models (LLMs), which shows more robust performance. Finally, we analyze remaining errors, revealing future directions for this task.

Zitieren

APA

Buaphet, W., Nguyen, T., Kondo, R., Kajiwara, T., Kim, Y., Lee, J., Lee, H., Lovenia, H., Limkonchotiwat, P., Nutanong, S., & Goot, R. V. D. (2026). MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages. ACM Transactions on Asian and Low-Resource Language Information Processing, 25(7), 1-14. https://doi.org/10.1145/3812651

Vancouver

Buaphet W, Nguyen T, Kondo R, Kajiwara T, Kim Y, Lee J, et al. MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages. ACM Transactions on Asian and Low-Resource Language Information Processing. 2026;25(7):1-14. doi:10.1145/3812651

BibTeX
@article{Buaphet2026,
  author = {Buaphet, Weerayut and Nguyen, Thanh-Nhi and Kondo, Risa and Kajiwara, Tomoyuki and Kim, Yumin and Lee, Jimin and Lee, Hwanhee and Lovenia, Holy and Limkonchotiwat, Peerat and Nutanong, Sarana and Goot, Rob Van der},
  title = {MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages},
  journal = {ACM Transactions on Asian and Low-Resource Language Information Processing},
  year = {2026},
  volume = {25},
  number = {7},
  pages = {1--14},
  doi = {10.1145/3812651}
}
Kostenlos starten →