EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model

Kanchon Kanti Podder iD, Maymouna Ezeddin, Muhammad E H Chowdhury iD, Md Shaheenur Islam Sumon iD, Anas M Tahir iD, Mohamed Arselene Ayari iD, Proma Dutta, Amith Khandakar iD, Zaid Bin Mahbub iD, Muhammad Abdul Kadir iD

Sensors · 14.08.2023 · 23 (16) · 7156

DOI: 10.3390/s23167156 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 14.08.2023. Verfasst von Kanchon Kanti Podder und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Physics & Technology, University of Dhaka, Dhaka 1000, Bangladesh. Hauptthemen laut MEDLINE: Deep Learning. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Every one of us has a unique manner of communicating to explore the world, and such communication helps to interpret life. Sign language is the popular language of communication for hearing and speech-disabled people. When a sign language user interacts with a non-sign language user, it becomes difficult for a signer to express themselves to another person. A sign language recognition system can help a signer to interpret the sign of a non-sign language user. This study presents a sign language recognition system that is capable of recognizing Arabic Sign Language from recorded RGB videos. To achieve this, two datasets were considered, such as (1) the raw dataset and (2) the face–hand region-based segmented dataset produced from the raw dataset. Moreover, operational layer-based multi-layer perceptron “SelfMLP” is proposed in this study to build CNN-LSTM-SelfMLP models for Arabic Sign Language recognition. MobileNetV2 and ResNet18-based CNN backbones and three SelfMLPs were used to construct six different models of CNN-LSTM-SelfMLP architecture for performance comparison of Arabic Sign Language recognition. This study examined the signer-independent mode to deal with real-time application circumstances. As a result, MobileNetV2-LSTM-SelfMLP on the segmented dataset achieved the best accuracy of 87.69% with 88.57% precision, 87.69% recall, 87.72% F1 score, and 99.75% specificity. Overall, face–hand region-based segmentation and SelfMLP-infused MobileNetV2-LSTM-SelfMLP surpassed the previous findings on Arabic Sign Language recognition by 10.970% accuracy.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Deep LearningHumansLanguageSign LanguageCommunicationRecognition, Psychology

Stichwörter der Autor:innen

Arabic Sign LanguageMediaPipedeep learningdynamic sign languagesegmentation

Autor:innen und Institutionen

  • Department of Biomedical Physics & Technology, University of Dhaka, Dhaka 1000, Bangladesh.
  • Department of Computer Science, Hamad Bin Khalifa University, Doha 34110, Qatar.
  • Department of Electrical Engineering, Qatar University, Doha 2713, Qatar.
  • Department of Biomedical Engineering, Military Institute of Science and Technology (MIST), Dhaka 1216, Bangladesh.
  • Department of Civil and Architectural Engineering, Qatar University, Doha 2713, Qatar.
  • Department of Electrical& Electronic Engineering, Chittagong University of Engineering & Technology, Chittagong 4349, Bangladesh.

ORCID-Profile: Kanchon Kanti Podder, Muhammad E H Chowdhury, Md Shaheenur Islam Sumon, Anas M Tahir, Mohamed Arselene Ayari, Amith Khandakar, Zaid Bin Mahbub, Muhammad Abdul Kadir

Zitieren

APA

Podder, K. K., Ezeddin, M., Chowdhury, M. E. H., Sumon, M. S. I., Tahir, A. M., Ayari, M. A., Dutta, P., Khandakar, A., Mahbub, Z. B., & Kadir, M. A. (2023). Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model. Sensors, 23(16), 7156. https://doi.org/10.3390/s23167156

Vancouver

Podder KK, Ezeddin M, Chowdhury MEH, Sumon MSI, Tahir AM, Ayari MA, et al. Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model. Sensors. 2023;23(16):7156. doi:10.3390/s23167156

BibTeX
@article{Podder2023,
  author = {Podder, Kanchon Kanti and Ezeddin, Maymouna and Chowdhury, Muhammad E H and Sumon, Md Shaheenur Islam and Tahir, Anas M and Ayari, Mohamed Arselene and Dutta, Proma and Khandakar, Amith and Mahbub, Zaid Bin and Kadir, Muhammad Abdul},
  title = {Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {16},
  pages = {7156},
  doi = {10.3390/s23167156}
}
Kostenlos starten →