EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

HGR-ViT: Hand Gesture Recognition with Vision Transformer

Chun Keat Tan, Kian Ming Lim iD, Roy Kwang Yang Chang, Chin Poo Lee iD, Ali Alqahtani iD

Sensors · 14.06.2023 · 23 (12) · 5555

DOI: 10.3390/s23125555 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 14.06.2023. Verfasst von Chun Keat Tan und 4 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Melaka 75450, Malaysia. Hauptthemen laut MEDLINE: Gestures, Pattern Recognition, Automated. Die Arbeit wurde bisher 48-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Telekom Malaysia Berhad (Malaysia), King Khalid University, Telekom Malaysia Research & Development.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Hand gesture recognition (HGR) is a crucial area of research that enhances communication by overcoming language barriers and facilitating human-computer interaction. Although previous works in HGR have employed deep neural networks, they fail to encode the orientation and position of the hand in the image. To address this issue, this paper proposes HGR-ViT, a Vision Transformer (ViT) model with an attention mechanism for hand gesture recognition. Given a hand gesture image, it is first split into fixed size patches. Positional embedding is added to these embeddings to form learnable vectors that capture the positional information of the hand patches. The resulting sequence of vectors are then served as the input to a standard Transformer encoder to obtain the hand gesture representation. A multilayer perceptron head is added to the output of the encoder to classify the hand gesture to the correct class. The proposed HGR-ViT obtains an accuracy of 99.98%, 99.36% and 99.85% for the American Sign Language (ASL) dataset, ASL with Digits dataset, and National University of Singapore (NUS) hand gesture dataset, respectively.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

GesturesPattern Recognition, AutomatedHumansNeural Networks, ComputerUpper ExtremitySign LanguageHand

Stichwörter der Autor:innen

ViTattentionhand gesture recognitionsign language recognitionvision transformer

Autor:innen und Institutionen

  • Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Melaka 75450, Malaysia.
  • Department of Computer Science, King Khalid University, Abha 61421, Saudi Arabia.
  • Center for Artificial Intelligence (CAI), King Khalid University, Abha 61421, Saudi Arabia.

ORCID-Profile: Kian Ming Lim, Chin Poo Lee, Ali Alqahtani

Förderung

  • Telekom Malaysia Berhad (Malaysia)
  • King Khalid University
  • Telekom Malaysia Research & Development
  • Deanship of Scientific Research, King Khalid University, Saudi Arabia

Zitieren

APA

Tan, C. K., Lim, K. M., Chang, R. K. Y., Lee, C. P., & Alqahtani, A. (2023). HGR-ViT: Hand Gesture Recognition with Vision Transformer. Sensors, 23(12), 5555. https://doi.org/10.3390/s23125555

Vancouver

Tan CK, Lim KM, Chang RKY, Lee CP, Alqahtani A. HGR-ViT: Hand Gesture Recognition with Vision Transformer. Sensors. 2023;23(12):5555. doi:10.3390/s23125555

BibTeX
@article{Tan2023,
  author = {Tan, Chun Keat and Lim, Kian Ming and Chang, Roy Kwang Yang and Lee, Chin Poo and Alqahtani, Ali},
  title = {HGR-ViT: Hand Gesture Recognition with Vision Transformer},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {12},
  pages = {5555},
  doi = {10.3390/s23125555}
}
Kostenlos starten →