UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals
Sensors · 2023
DOI: 10.3390/s23041790 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von Yuejiao Wang und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China. Hauptthemen laut MEDLINE: Ultrasonics, COVID-19. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Zhanjun Hao, National Natural Science Foundation of China, Key Science and Technology Support Program of Gansu Province.
Abstract (Originalsprache)
With the global spread of the novel coronavirus, avoiding human-to-human contact has become an effective way to cut off the spread of the virus. Therefore, contactless gesture recognition becomes an effective means to reduce the risk of contact infection in outbreak prevention and control. However, the recognition of everyday behavioral sign language of a certain population of deaf people presents a challenge to sensing technology. Ubiquitous acoustics offer new ideas on how to perceive everyday behavior. The advantages of a low sampling rate, slow propagation speed, and easy access to the equipment have led to the widespread use of acoustic signal-based gesture recognition sensing technology. Therefore, this paper proposed a contactless gesture and sign language behavior sensing method based on ultrasonic signals—UltrasonicGS. The method used Generative Adversarial Network (GAN)-based data augmentation techniques to expand the dataset without human intervention and improve the performance of the behavior recognition model. In addition, to solve the problem of inconsistent length and difficult alignment of input and output sequences of continuous gestures and sign language gestures, we added the Connectionist Temporal Classification (CTC) algorithm after the CRNN network. Additionally, the architecture can achieve better recognition of sign language behaviors of certain people, filling the gap of acoustic-based perception of Chinese sign language. We have conducted extensive experiments and evaluations of UltrasonicGS in a variety of real scenarios. The experimental results showed that UltrasonicGS achieved a combined recognition rate of 98.8% for 15 single gestures and an average correct recognition rate of 92.4% and 86.3% for six sets of continuous gestures and sign language gestures, respectively. As a result, our proposed method provided a low-cost and highly robust solution for avoiding human-to-human contact.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Computer Science and Engineering, Northwest Normal University, Lanzhou 730070, China.
- Gansu Province Internet of Things Engineering Research Center, Lanzhou 730070, China.
ORCID-Profile: Yuejiao Wang, Zhanjun Hao
Förderung
- Zhanjun Hao
- National Natural Science Foundation of China
- Key Science and Technology Support Program of Gansu Province
- 2019 Chinese Academy of Sciences “Light of the West” Talent Program, Science and Technology Innovation Project of Gansu Province
- 2019 Lanzhou City Science and Technology Plan Project
- 2020 Lanzhou City Talent Innovation and Entrepreneurship Project
- Gansu Provincial Department of Education: Industry Support Program Project
Zitieren
APA
Wang, Y., Hao, Z., Dang, X., Zhang, Z., & Li, M. (2023). UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals. Sensors. https://doi.org/10.3390/s23041790
Vancouver
Wang Y, Hao Z, Dang X, Zhang Z, Li M. UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals. Sensors. 2023. doi:10.3390/s23041790
BibTeX
@article{Wang2023,
author = {Wang, Yuejiao and Hao, Zhanjun and Dang, Xiaochao and Zhang, Zhenyi and Li, Mengqiao},
title = {UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals},
journal = {Sensors},
year = {2023},
doi = {10.3390/s23041790}
}
EUVIMED