EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques

Dana El-Rushaidat iD, Nour Almohammad, Raine Yeh iD, Kinda Fayyad

Journal of Imaging · 20.04.2026 · 12 (4) · 177

DOI: 10.3390/jimaging12040177 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Imaging, online erschienen am 20.04.2026. Verfasst von Dana El-Rushaidat und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Jordan University of Science and Technology, Irbid 22110, Jordan. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper addresses the critical communication barrier experienced by deaf and hearing-impaired individuals in the Arab world through the development of an affordable, video-based Arabic Sign Language (ArSL) recognition system. Designed for broad accessibility, the system eliminates specialized hardware by leveraging standard mobile or laptop cameras. Our methodology employs Mediapipe for real-time extraction of hand, face, and pose landmarks from video streams. These anatomical features are then processed by a hybrid deep learning model integrating Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), specifically Bidirectional Long Short-Term Memory (BiLSTM) layers. The CNN component captures spatial features, such as intricate hand shapes and body movements, within individual frames. Concurrently, BiLSTMs model long-term temporal dependencies and motion trajectories across consecutive frames. This integrated CNN-BiLSTM architecture is critical for generating a comprehensive spatiotemporal representation, enabling accurate differentiation of complex signs where meaning relies on both static gestures and dynamic transitions, thus preventing misclassification that CNN-only or RNN-only models would incur. Rigorously evaluated on the author-created JUST-SL dataset and the publicly available KArSL dataset, the system achieved 96% overall accuracy for JUST-SL and an impressive 99% for KArSL. These results demonstrate the system’s superior accuracy compared to previous research, particularly for recognizing full Arabic words, thereby significantly enhancing communication accessibility for the deaf and hearing-impaired community.

Schlagwörter

Stichwörter der Autor:innen

Arabic Sign Language Recognition (ArSLR)Deep LearningMediapipe libraryMultimediaSign Language (SL)

Autor:innen und Institutionen

  • Department of Computer Science, Jordan University of Science and Technology, Irbid 22110, Jordan.
  • Google, New York City, NY 10011, USA.

ORCID-Profile: Dana El-Rushaidat, Raine Yeh

Zitieren

APA

El-Rushaidat, D., Almohammad, N., Yeh, R., & Fayyad, K. (2026). Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques. Journal of Imaging, 12(4), 177. https://doi.org/10.3390/jimaging12040177

Vancouver

El-Rushaidat D, Almohammad N, Yeh R, Fayyad K. Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques. Journal of Imaging. 2026;12(4):177. doi:10.3390/jimaging12040177

BibTeX
@article{ElRushaidat2026,
  author = {El-Rushaidat, Dana and Almohammad, Nour and Yeh, Raine and Fayyad, Kinda},
  title = {Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques},
  journal = {Journal of Imaging},
  year = {2026},
  volume = {12},
  number = {4},
  pages = {177},
  doi = {10.3390/jimaging12040177}
}
Kostenlos starten →