EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People

Md Johir Raihan iD, Mainul Islam Labib, Abdullah Al Jaid Jim iD, Jun Jiat Tiang iD, Uzzal Biswas iD, Abdullah-Al Nahid iD

Sensors · 2024

DOI: 10.3390/s24165351 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Md Johir Raihan und 5 weiteren Autor:innen. Beteiligte Einrichtung: Electronics and Communication Engineering Discipline, Khulna University, Khulna 9208, Bangladesh. Hauptthemen laut MEDLINE: Sign Language, Machine Learning, Deafness. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Management Centre, Multimedia University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sign language is undoubtedly a common way of communication among deaf and non-verbal people. But it is not common among hearing people to use sign language to express feelings or share information in everyday life. Therefore, a significant communication gap exists between deaf and hearing individuals, despite both groups experiencing similar emotions and sentiments. In this paper, we developed a convolutional neural network–squeeze excitation network to predict the sign language signs and developed a smartphone application to provide access to the ML model to use it. The SE block provides attention to the channel of the image, thus improving the performance of the model. On the other hand, the smartphone application brings the ML model close to people so that everyone can benefit from it. In addition, we used the Shapley additive explanation to interpret the black box nature of the ML model and understand the models working from within. Using our ML model, we achieved an accuracy of 99.86% on the KU-BdSL dataset. The SHAP analysis shows that the model primarily relies on hand-related visual cues to predict sign language signs, aligning with human communication patterns.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Sign LanguageMachine LearningDeafnessHumansNeural Networks, ComputerSmartphonePersons with Hearing Disabilities

Stichwörter der Autor:innen

Bengali sign language (BdSL)SHAPconvolutional neural network (CNN)squeeze excitation (SE)

Autor:innen und Institutionen

  • Electronics and Communication Engineering Discipline, Khulna University, Khulna 9208, Bangladesh.
  • Department of Electrical and Electronics Engineering, Trust University, Barishal 8200, Bangladesh.
  • Centre For Wireless Technology (CWT), Faculty of Engineering, Multimedia University, Cyberjaya 63100, Malaysia.

ORCID-Profile: Md Johir Raihan, Abdullah Al Jaid Jim, Jun Jiat Tiang, Uzzal Biswas, Abdullah-Al Nahid

Förderung

  • Research Management Centre, Multimedia University

Zitieren

APA

Raihan, M. J., Labib, M. I., Jim, A. A. J., Tiang, J. J., Biswas, U., & Nahid, A. A. (2024). Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People. Sensors. https://doi.org/10.3390/s24165351

Vancouver

Raihan MJ, Labib MI, Jim AAJ, Tiang JJ, Biswas U, Nahid AA. Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People. Sensors. 2024. doi:10.3390/s24165351

BibTeX
@article{Raihan2024,
  author = {Raihan, Md Johir and Labib, Mainul Islam and Jim, Abdullah Al Jaid and Tiang, Jun Jiat and Biswas, Uzzal and Nahid, Abdullah-Al},
  title = {Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24165351}
}
Kostenlos starten →