A Performance Comparison of Japanese Sign Language Recognition with ViT and CNN Using Angular Features
Applied Sciences · 11.04.2024 · 14 (8) · 3228
DOI: 10.3390/app14083228 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 11.04.2024. Verfasst von Tamon Kondo und 4 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Co-G.E.I. (Cooperative Good Educational Innovation) Challenge 2023 of Tokyo Polytechnic University.
Abstract (Originalsprache)
In recent years, developments in deep learning technology have driven significant advancements in research aimed at facilitating communication with individuals who have hearing impairments. The focus has been on enhancing automatic recognition and translation systems for sign language. This study proposes a novel approach using a vision transformer (ViT) for recognizing Japanese Sign Language. Our method employs a pose estimation library, MediaPipe, to extract the positional coordinates of each finger joint within video frames and generate one-dimensional angular feature data from these coordinates. Then, the code arranges these feature data in a temporal sequence to form a two-dimensional input vector for the ViT model. To determine the optimal configuration, this study evaluated recognition accuracy by manipulating the number of encoder layers within the ViT model and compared against traditional convolutional neural network (CNN) models to evaluate its effectiveness. The experimental results showed 99.7% accuracy for the method using the ViT model and 99.3% for the results using the CNN. We demonstrated the efficacy of our approach through real-time recognition experiments using Japanese sign language videos.
Förderung
- Co-G.E.I. (Cooperative Good Educational Innovation) Challenge 2023 of Tokyo Polytechnic University
Zitieren
APA
Kondo, T., Narumi, S., He, Z., Shin, D., & Kang, Y. (2024). A Performance Comparison of Japanese Sign Language Recognition with ViT and CNN Using Angular Features. Applied Sciences, 14(8), 3228. https://doi.org/10.3390/app14083228
Vancouver
Kondo T, Narumi S, He Z, Shin D, Kang Y. A Performance Comparison of Japanese Sign Language Recognition with ViT and CNN Using Angular Features. Applied Sciences. 2024;14(8):3228. doi:10.3390/app14083228
BibTeX
@article{Kondo2024,
author = {Kondo, Tamon and Narumi, Sakura and He, Zixun and Shin, Duk and Kang, Yousun},
title = {A Performance Comparison of Japanese Sign Language Recognition with ViT and CNN Using Angular Features},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {8},
pages = {3228},
doi = {10.3390/app14083228}
}
EUVIMED