Word prediction using Dynamic Skip Connections along with ARABERT and LSTM in Arabic Language
ASEAN Journal on Science and Technology for Development · 01.01.2024 · 41 (1)
DOI: 10.61931/2224-9028.1569 ↗Kurz gefasst
Fachartikel in der Zeitschrift ASEAN Journal on Science and Technology for Development, online erschienen am 01.01.2024. Verfasst von Ahad Almalki und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
Natural Language Generation (NLG) plays a crucial role in modern digital tools, including chatbots, virtual support, content suggestions, and tailored marketing, making bots more responsive and reducing the need for human staff. While there's much research on NLG for languages like English, languages like Arabic, Urdu, and Chinese still face challenges. This study examines Arabic NLG's unique aspects, dialects, and word variations. With around 420 million Arabic speakers globally, it's crucial to advance NLG for this language. We compared three models: Long Short-Term Memory (LSTM), a mix of Bidirectional Encoder Representations from Transformers (BERT) and LSTM, and a version that adds Dynamic Skip Connection (DSC). Our aim is to find the best model for predicting Arabic words with the least mistakes. In our experiment, we found that adding DSC wasn't beneficial. However, combining BERT and LSTM with an attention mechanism reduced loss and favorable perplexity values.
Zitieren
APA
Almalki, A., Kateb, F., & Mosli, R. (2024). Word prediction using Dynamic Skip Connections along with ARABERT and LSTM in Arabic Language. ASEAN Journal on Science and Technology for Development, 41(1). https://doi.org/10.61931/2224-9028.1569
Vancouver
Almalki A, Kateb F, Mosli R. Word prediction using Dynamic Skip Connections along with ARABERT and LSTM in Arabic Language. ASEAN Journal on Science and Technology for Development. 2024;41(1). doi:10.61931/2224-9028.1569
BibTeX
@article{Almalki2024,
author = {Almalki, Ahad and Kateb, Faris and Mosli, Rayan},
title = {Word prediction using Dynamic Skip Connections along with ARABERT and LSTM in Arabic Language},
journal = {ASEAN Journal on Science and Technology for Development},
year = {2024},
volume = {41},
number = {1},
doi = {10.61931/2224-9028.1569}
}
EUVIMED