Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People
Sensors · 2024
DOI: 10.3390/s24165351 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Md Johir Raihan und 5 weiteren Autor:innen. Beteiligte Einrichtung: Electronics and Communication Engineering Discipline, Khulna University, Khulna 9208, Bangladesh. Hauptthemen laut MEDLINE: Sign Language, Machine Learning, Deafness. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Management Centre, Multimedia University.
Abstract (Originalsprache)
Sign language is undoubtedly a common way of communication among deaf and non-verbal people. But it is not common among hearing people to use sign language to express feelings or share information in everyday life. Therefore, a significant communication gap exists between deaf and hearing individuals, despite both groups experiencing similar emotions and sentiments. In this paper, we developed a convolutional neural network–squeeze excitation network to predict the sign language signs and developed a smartphone application to provide access to the ML model to use it. The SE block provides attention to the channel of the image, thus improving the performance of the model. On the other hand, the smartphone application brings the ML model close to people so that everyone can benefit from it. In addition, we used the Shapley additive explanation to interpret the black box nature of the ML model and understand the models working from within. Using our ML model, we achieved an accuracy of 99.86% on the KU-BdSL dataset. The SHAP analysis shows that the model primarily relies on hand-related visual cues to predict sign language signs, aligning with human communication patterns.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Electronics and Communication Engineering Discipline, Khulna University, Khulna 9208, Bangladesh.
- Department of Electrical and Electronics Engineering, Trust University, Barishal 8200, Bangladesh.
- Centre For Wireless Technology (CWT), Faculty of Engineering, Multimedia University, Cyberjaya 63100, Malaysia.
ORCID-Profile: Md Johir Raihan, Abdullah Al Jaid Jim, Jun Jiat Tiang, Uzzal Biswas, Abdullah-Al Nahid
Förderung
- Research Management Centre, Multimedia University
Zitieren
APA
Raihan, M. J., Labib, M. I., Jim, A. A. J., Tiang, J. J., Biswas, U., & Nahid, A. A. (2024). Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People. Sensors. https://doi.org/10.3390/s24165351
Vancouver
Raihan MJ, Labib MI, Jim AAJ, Tiang JJ, Biswas U, Nahid AA. Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People. Sensors. 2024. doi:10.3390/s24165351
BibTeX
@article{Raihan2024,
author = {Raihan, Md Johir and Labib, Mainul Islam and Jim, Abdullah Al Jaid and Tiang, Jun Jiat and Biswas, Uzzal and Nahid, Abdullah-Al},
title = {Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24165351}
}
EUVIMED