HGR-ViT: Hand Gesture Recognition with Vision Transformer
Sensors · 14.06.2023 · 23 (12) · 5555
DOI: 10.3390/s23125555 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 14.06.2023. Verfasst von Chun Keat Tan und 4 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Melaka 75450, Malaysia. Hauptthemen laut MEDLINE: Gestures, Pattern Recognition, Automated. Die Arbeit wurde bisher 48-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Telekom Malaysia Berhad (Malaysia), King Khalid University, Telekom Malaysia Research & Development.
Abstract (Originalsprache)
Hand gesture recognition (HGR) is a crucial area of research that enhances communication by overcoming language barriers and facilitating human-computer interaction. Although previous works in HGR have employed deep neural networks, they fail to encode the orientation and position of the hand in the image. To address this issue, this paper proposes HGR-ViT, a Vision Transformer (ViT) model with an attention mechanism for hand gesture recognition. Given a hand gesture image, it is first split into fixed size patches. Positional embedding is added to these embeddings to form learnable vectors that capture the positional information of the hand patches. The resulting sequence of vectors are then served as the input to a standard Transformer encoder to obtain the hand gesture representation. A multilayer perceptron head is added to the output of the encoder to classify the hand gesture to the correct class. The proposed HGR-ViT obtains an accuracy of 99.98%, 99.36% and 99.85% for the American Sign Language (ASL) dataset, ASL with Digits dataset, and National University of Singapore (NUS) hand gesture dataset, respectively.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Melaka 75450, Malaysia.
- Department of Computer Science, King Khalid University, Abha 61421, Saudi Arabia.
- Center for Artificial Intelligence (CAI), King Khalid University, Abha 61421, Saudi Arabia.
ORCID-Profile: Kian Ming Lim, Chin Poo Lee, Ali Alqahtani
Förderung
- Telekom Malaysia Berhad (Malaysia)
- King Khalid University
- Telekom Malaysia Research & Development
- Deanship of Scientific Research, King Khalid University, Saudi Arabia
Zitieren
APA
Tan, C. K., Lim, K. M., Chang, R. K. Y., Lee, C. P., & Alqahtani, A. (2023). HGR-ViT: Hand Gesture Recognition with Vision Transformer. Sensors, 23(12), 5555. https://doi.org/10.3390/s23125555
Vancouver
Tan CK, Lim KM, Chang RKY, Lee CP, Alqahtani A. HGR-ViT: Hand Gesture Recognition with Vision Transformer. Sensors. 2023;23(12):5555. doi:10.3390/s23125555
BibTeX
@article{Tan2023,
author = {Tan, Chun Keat and Lim, Kian Ming and Chang, Roy Kwang Yang and Lee, Chin Poo and Alqahtani, Ali},
title = {HGR-ViT: Hand Gesture Recognition with Vision Transformer},
journal = {Sensors},
year = {2023},
volume = {23},
number = {12},
pages = {5555},
doi = {10.3390/s23125555}
}
EUVIMED