EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An Intelligent Real-Time System for Sentence-Level Recognition of Continuous Saudi Sign Language Using Landmark-Based Temporal Modeling

Adel BenAbdennour iD, Mohammed Mukhtar iD, Osama Almolike, Bilal A Khawaja iD, Abdulmajeed M Alenezi iD

Sensors · 2026

DOI: 10.3390/s26051652 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Adel BenAbdennour und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical Engineering, Faculty of Engineering, Islamic University of Madinah, Madinah 42351, Saudi Arabia. Hauptthemen laut MEDLINE: Sign Language. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Salman Center for Disability Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A persistent challenge for Deaf and Hard-of-Hearing individuals is the communication gap between sign language users and the hearing community, particularly in regions with limited automated translation resources. In Saudi Arabia, this gap is amplified by the reliance on Saudi Sign Language (SSL) and the scarcity of real-time, sentence-level translation systems. This paper presents a real-time system for sentence-level recognition of continuous SSL and direct mapping to natural spoken Arabic. The proposed system operates end-to-end on live video streams or pre-recorded content, extracting spatio-temporal landmark features using the MediaPipe Holistic framework. For classification, the input feature vector consists of 225 features derived from hand and body pose landmarks. These features are processed by a Bidirectional Long Short-Term Memory (BiLSTM) network trained on the ArabSign (ArSL) dataset to perform direct sentence-level classification over a vocabulary of 50 continuous Arabic sign language sentences, supported by an idle-based segmentation mechanism that enables natural, uninterrupted signing. Experimental evaluation demonstrates robust generalization: under a Leave-One-Signer-Out (LOSO) cross-validation protocol, the model attains a mean sentence-level accuracy of 94.2%, outperforming the fixed signer-independent split baseline of 92.07%, while maintaining real-time performance suitable for interactive use. To enhance linguistic fluency, an optional post-recognition refinement stage is incorporated using a large language model (LLM), followed by text-to-speech synthesis to produce audible Arabic output; this refinement operates strictly as post-processing and is not included in the reported recognition accuracy metrics. The results demonstrate that direct sentence-level modeling, combined with landmark-based feature extraction and real-time segmentation, provides an effective and practical solution for continuous SSL sentence recognition in real-time.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Sign LanguageHumansSaudi ArabiaLong Short Term MemoryIntelligent SystemsDeafness

Stichwörter der Autor:innen

Bidirectional Long Short-Term MemorySaudi Sign Languageartificial intelligenceassistive technologiesdeep learningdisabilitylarge language modelsentence-level recognitionsign languagetemporal modeling

Autor:innen und Institutionen

  • Department of Electrical Engineering, Faculty of Engineering, Islamic University of Madinah, Madinah 42351, Saudi Arabia.

ORCID-Profile: Adel BenAbdennour, Mohammed Mukhtar, Bilal A Khawaja, Abdulmajeed M Alenezi

Förderung

  • King Salman Center for Disability Research

Zitieren

APA

BenAbdennour, A., Mukhtar, M., Almolike, O., Khawaja, B. A., & Alenezi, A. M. (2026). An Intelligent Real-Time System for Sentence-Level Recognition of Continuous Saudi Sign Language Using Landmark-Based Temporal Modeling. Sensors. https://doi.org/10.3390/s26051652

Vancouver

BenAbdennour A, Mukhtar M, Almolike O, Khawaja BA, Alenezi AM. An Intelligent Real-Time System for Sentence-Level Recognition of Continuous Saudi Sign Language Using Landmark-Based Temporal Modeling. Sensors. 2026. doi:10.3390/s26051652

BibTeX
@article{BenAbdennour2026,
  author = {BenAbdennour, Adel and Mukhtar, Mohammed and Almolike, Osama and Khawaja, Bilal A and Alenezi, Abdulmajeed M},
  title = {An Intelligent Real-Time System for Sentence-Level Recognition of Continuous Saudi Sign Language Using Landmark-Based Temporal Modeling},
  journal = {Sensors},
  year = {2026},
  doi = {10.3390/s26051652}
}
Kostenlos starten →