EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Transformer Model for Direct Translation of Spoken Text to Arabic Sign Language (ArSL)

Samah Abbas, Dimah Alahmadi, Hassanin Al-Barhamtoshy

ACM Transactions on Asian and Low-Resource Language Information Processing · 30.06.2026 · 25 (6) · 1-33

DOI: 10.1145/3815422 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Asian and Low-Resource Language Information Processing, online erschienen am 08.06.2026. Verfasst von Samah Abbas und 2 weiteren Autor:innen. Die Arbeit nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: KAU Endowment (WAQF) at King Abdulaziz University, Jeddah, Saudi Arabia.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Arabic Sign Language (ArSL) is a formal language used by Deaf and Hard of Hearing communities. It helps them understand Arabic spoken by hearing people, particularly in formal speech, such as news or sermons. The production of sign language systems can support communication by translating spoken languages into ArSL videos. However, current systems often fail to meet user needs due to poor translation of grammatical structures and limited ability to capture the textual meaning of speech. This research focused on a neural machine translation (NMT) approach to convert spoken Arabic texts into Arabic Sign Language (ArSL) using a transformer-based model to overcome these difficulties. The ArSL video of the sign is produced as a series of 2D skeletal poses. The proposed methodology is based on an encoder-decoder architecture: texts are represented by a transformer model, and a posture decoder generates gestures. The model is evaluated using quantitative criteria, such as mean square error (MSE) and the dynamic time warping (DTW) measure, as well as qualitative evaluations by sign language experts and users from the Deaf and Hard of Hearing community. The study uses Friday sermon data as a reference to build the translation model and applies optimization techniques such as temporal modulation, noise addition, and resizing to enhance performance. The results show that the enhanced model improves the quality of gesture representation, reducing DTW by 43.4% relative to the baseline in the non-continuous ArSL test. On the other hand, in the Continuous ArSL test using Short Sermon, the DTW is reduced by around 2.25%. In addition, the enhanced model received high user acceptance, with 75.2% satisfaction. Moreover, unified qualitative evaluation criteria for sign language production are proposed, including clarity, accuracy, consistency, and naturalness, which could serve as a basis for future evaluations in this field. These criteria are used to evaluate the generated ArSL and measure user satisfaction.

Förderung

  • KAU Endowment (WAQF) at King Abdulaziz University, Jeddah, Saudi Arabia

Zitieren

APA

Abbas, S., Alahmadi, D., & Al-Barhamtoshy, H. (2026). Transformer Model for Direct Translation of Spoken Text to Arabic Sign Language (ArSL). ACM Transactions on Asian and Low-Resource Language Information Processing, 25(6), 1-33. https://doi.org/10.1145/3815422

Vancouver

Abbas S, Alahmadi D, Al-Barhamtoshy H. Transformer Model for Direct Translation of Spoken Text to Arabic Sign Language (ArSL). ACM Transactions on Asian and Low-Resource Language Information Processing. 2026;25(6):1-33. doi:10.1145/3815422

BibTeX
@article{Abbas2026,
  author = {Abbas, Samah and Alahmadi, Dimah and Al-Barhamtoshy, Hassanin},
  title = {Transformer Model for Direct Translation of Spoken Text to Arabic Sign Language (ArSL)},
  journal = {ACM Transactions on Asian and Low-Resource Language Information Processing},
  year = {2026},
  volume = {25},
  number = {6},
  pages = {1--33},
  doi = {10.1145/3815422}
}
Kostenlos starten →