EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System

Yutong Gu, Hiromasa Oku iD, Masahiro Todoh iD

Sensors · 2024

DOI: 10.3390/s24020453 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Yutong Gu und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Informatics, Gunma University, Kiryu 3768515, Japan. Hauptthemen laut MEDLINE: Sign Language, Wearable Electronic Devices. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Gunma University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sign language is designed as a natural communication method to convey messages among the deaf community. In the study of sign language recognition through wearable sensors, the data sources are limited, and the data acquisition process is complex. This research aims to collect an American sign language dataset with a wearable inertial motion capture system and realize the recognition and end-to-end translation of sign language sentences with deep learning models. In this work, a dataset consisting of 300 commonly used sentences is gathered from 3 volunteers. In the design of the recognition network, the model mainly consists of three layers: convolutional neural network, bi-directional long short-term memory, and connectionist temporal classification. The model achieves accuracy rates of 99.07% in word-level evaluation and 97.34% in sentence-level evaluation. In the design of the translation network, the encoder-decoder structured model is mainly based on long short-term memory with global attention. The word error rate of end-to-end translation is 16.63%. The proposed method has the potential to recognize more sign language sentences with reliable inertial data from the device.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Sign LanguageWearable Electronic DevicesHumansUnited StatesMotion CaptureNeuronsPerception

Stichwörter der Autor:innen

American sign languagedeep learning modelswearable inertial sensors

Autor:innen und Institutionen

  • Faculty of Informatics, Gunma University, Kiryu 3768515, Japan.
  • Graduate School of Engineering, Hokkaido University, Sapporo 0608628, Japan.
  • Faculty of Engineering, Hokkaido University, Sapporo 0608628, Japan.

ORCID-Profile: Hiromasa Oku, Masahiro Todoh

Förderung

  • Gunma University

Zitieren

APA

Gu, Y., Oku, H., & Todoh, M. (2024). American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System. Sensors. https://doi.org/10.3390/s24020453

Vancouver

Gu Y, Oku H, Todoh M. American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System. Sensors. 2024. doi:10.3390/s24020453

BibTeX
@article{Gu2024,
  author = {Gu, Yutong and Oku, Hiromasa and Todoh, Masahiro},
  title = {American Sign Language Recognition and Translation Using Perception Neuron Wearable Inertial Motion Capture System},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24020453}
}
Kostenlos starten →