Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques
Journal of Imaging · 20.04.2026 · 12 (4) · 177
DOI: 10.3390/jimaging12040177 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Imaging, online erschienen am 20.04.2026. Verfasst von Dana El-Rushaidat und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Jordan University of Science and Technology, Irbid 22110, Jordan. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This paper addresses the critical communication barrier experienced by deaf and hearing-impaired individuals in the Arab world through the development of an affordable, video-based Arabic Sign Language (ArSL) recognition system. Designed for broad accessibility, the system eliminates specialized hardware by leveraging standard mobile or laptop cameras. Our methodology employs Mediapipe for real-time extraction of hand, face, and pose landmarks from video streams. These anatomical features are then processed by a hybrid deep learning model integrating Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), specifically Bidirectional Long Short-Term Memory (BiLSTM) layers. The CNN component captures spatial features, such as intricate hand shapes and body movements, within individual frames. Concurrently, BiLSTMs model long-term temporal dependencies and motion trajectories across consecutive frames. This integrated CNN-BiLSTM architecture is critical for generating a comprehensive spatiotemporal representation, enabling accurate differentiation of complex signs where meaning relies on both static gestures and dynamic transitions, thus preventing misclassification that CNN-only or RNN-only models would incur. Rigorously evaluated on the author-created JUST-SL dataset and the publicly available KArSL dataset, the system achieved 96% overall accuracy for JUST-SL and an impressive 99% for KArSL. These results demonstrate the system’s superior accuracy compared to previous research, particularly for recognizing full Arabic words, thereby significantly enhancing communication accessibility for the deaf and hearing-impaired community.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science, Jordan University of Science and Technology, Irbid 22110, Jordan.
- Google, New York City, NY 10011, USA.
ORCID-Profile: Dana El-Rushaidat, Raine Yeh
Zitieren
APA
El-Rushaidat, D., Almohammad, N., Yeh, R., & Fayyad, K. (2026). Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques. Journal of Imaging, 12(4), 177. https://doi.org/10.3390/jimaging12040177
Vancouver
El-Rushaidat D, Almohammad N, Yeh R, Fayyad K. Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques. Journal of Imaging. 2026;12(4):177. doi:10.3390/jimaging12040177
BibTeX
@article{ElRushaidat2026,
author = {El-Rushaidat, Dana and Almohammad, Nour and Yeh, Raine and Fayyad, Kinda},
title = {Video-Based Arabic Sign Language Recognition with Mediapipe and Deep Learning Techniques},
journal = {Journal of Imaging},
year = {2026},
volume = {12},
number = {4},
pages = {177},
doi = {10.3390/jimaging12040177}
}
EUVIMED