American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System
Sensors · 2024
DOI: 10.3390/s24020453 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Yutong Gu und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Informatics, Gunma University, Kiryu 3768515, Japan. Hauptthemen laut MEDLINE: Sign Language, Wearable Electronic Devices. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Gunma University.
Abstract (Originalsprache)
Sign language is designed as a natural communication method to convey messages among the deaf community. In the study of sign language recognition through wearable sensors, the data sources are limited, and the data acquisition process is complex. This research aims to collect an American sign language dataset with a wearable inertial motion capture system and realize the recognition and end-to-end translation of sign language sentences with deep learning models. In this work, a dataset consisting of 300 commonly used sentences is gathered from 3 volunteers. In the design of the recognition network, the model mainly consists of three layers: convolutional neural network, bi-directional long short-term memory, and connectionist temporal classification. The model achieves accuracy rates of 99.07% in word-level evaluation and 97.34% in sentence-level evaluation. In the design of the translation network, the encoder-decoder structured model is mainly based on long short-term memory with global attention. The word error rate of end-to-end translation is 16.63%. The proposed method has the potential to recognize more sign language sentences with reliable inertial data from the device.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Informatics, Gunma University, Kiryu 3768515, Japan.
- Graduate School of Engineering, Hokkaido University, Sapporo 0608628, Japan.
- Faculty of Engineering, Hokkaido University, Sapporo 0608628, Japan.
ORCID-Profile: Hiromasa Oku, Masahiro Todoh
Förderung
- Gunma University
Zitieren
APA
Gu, Y., Oku, H., & Todoh, M. (2024). American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System. Sensors. https://doi.org/10.3390/s24020453
Vancouver
Gu Y, Oku H, Todoh M. American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System. Sensors. 2024. doi:10.3390/s24020453
BibTeX
@article{Gu2024,
author = {Gu, Yutong and Oku, Hiromasa and Todoh, Masahiro},
title = {American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24020453}
}
EUVIMED