EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Deep Learning in Sign Language Recognition: A Hybrid Approach for the Recognition of Static and Dynamic Signs

Ahmed Mateen Buttar, Usama Ahmad, Abdu H. Gumaei, Adel Assiri, Muhammad Azeem Akbar, Bader Fahad Alkhamees

Mathematics · 30.08.2023 · 11 (17) · 3729

DOI: 10.3390/math11173729 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 30.08.2023. Verfasst von Ahmed Mateen Buttar und 5 weiteren Autor:innen. Die Arbeit wurde bisher 108-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Salman Center for Disability Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A speech impairment limits a person’s capacity for oral and auditory communication. A great improvement in communication between the deaf and the general public would be represented by a real-time sign language detector. This work proposes a deep learning-based algorithm that can identify words from a person’s gestures and detect them. There have been many studies on this topic, but the development of static and dynamic sign language recognition models is still a challenging area of research. The difficulty is in obtaining an appropriate model that addresses the challenges of continuous signs that are independent of the signer. Different signers’ speeds, durations, and many other factors make it challenging to create a model with high accuracy and continuity. For the accurate and effective recognition of signs, this study uses two different deep learning-based approaches. We create a real-time American Sign Language detector using the skeleton model, which reliably categorizes continuous signs in sign language in most cases using a deep learning approach. In the second deep learning approach, we create a sign language detector for static signs using YOLOv6. This application is very helpful for sign language users and learners to practice sign language in real time. After training both algorithms separately for static and continuous signs, we create a single algorithm using a hybrid approach. The proposed model, consisting of LSTM with MediaPipe holistic landmarks, achieves around 92% accuracy for different continuous signs, and the YOLOv6 model achieves 96% accuracy over different static signs. Throughout this study, we determine which approach is best for sequential movement detection and for the classification of different signs according to sign language and shows remarkable accuracy in real time.

Förderung

  • King Salman Center for Disability Research

Zitieren

APA

Buttar, A. M., Ahmad, U., Gumaei, A. H., Assiri, A., Akbar, M. A., & Alkhamees, B. F. (2023). Deep Learning in Sign Language Recognition: A Hybrid Approach for the Recognition of Static and Dynamic Signs. Mathematics, 11(17), 3729. https://doi.org/10.3390/math11173729

Vancouver

Buttar AM, Ahmad U, Gumaei AH, Assiri A, Akbar MA, Alkhamees BF. Deep Learning in Sign Language Recognition: A Hybrid Approach for the Recognition of Static and Dynamic Signs. Mathematics. 2023;11(17):3729. doi:10.3390/math11173729

BibTeX
@article{Buttar2023,
  author = {Buttar, Ahmed Mateen and Ahmad, Usama and Gumaei, Abdu H. and Assiri, Adel and Akbar, Muhammad Azeem and Alkhamees, Bader Fahad},
  title = {Deep Learning in Sign Language Recognition: A Hybrid Approach for the Recognition of Static and Dynamic Signs},
  journal = {Mathematics},
  year = {2023},
  volume = {11},
  number = {17},
  pages = {3729},
  doi = {10.3390/math11173729}
}
Kostenlos starten →