EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition

Pengcheng Han iD, Xin He iD, Takafumi Matsumaru iD, Vibekananda Dutta iD

Sensors · 2025

DOI: 10.3390/s25030702 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Pengcheng Han und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information, Production and System, Waseda University, Kitakyushu 808-0135, Japan. Hauptthemen laut MEDLINE: Gestures, Hand, Pattern Recognition, Automated. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Waseda University Grant for Special Research Project, Japan Society for the Promotion of Science, JSPS KAKENHI.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Manually crafted features often suffer from being subjective, having an inadequate accuracy, or lacking in robustness in recognition. Meanwhile, existing deep learning methods often overlook the structural and dynamic characteristics of the human hand, failing to fully explore the contextual information of joints in both the spatial and temporal domains. To effectively capture dependencies between the hand joints that are not adjacent but may have potential connections, it is essential to learn long-term relationships. This study proposes a skeleton-based hand gesture recognition framework, the ST-KT, a spatio-temporal graph convolution network, and a transformer with the Kolmogorov–Arnold Network (KAN) model. It incorporates spatio-temporal graph convolution network (ST-GCN) modules and a spatio-temporal transformer module with KAN (KAN–Transformer). ST-GCN modules, which include a spatial graph convolution network (SGCN) and a temporal convolution network (TCN), extract primary features from skeleton sequences by leveraging the strength of graph convolutional networks in the spatio-temporal domain. A spatio-temporal position embedding method integrates node features, enriching representations by including node identities and temporal information. The transformer layer includes a spatial KAN–Transformer (S-KT) and a temporal KAN–Transformer (T-KT), which further extract joint features by learning edge weights and node embeddings, providing richer feature representations and the capability for nonlinear modeling. We evaluated the performance of our method on two challenging skeleton-based dynamic gesture datasets: our method achieved an accuracy of 97.5% on the SHREC’17 track dataset and 94.3% on the DHG-14/28 dataset. These results demonstrate that our proposed method, ST-KT, effectively captures dynamic skeleton changes and complex joint relationships.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

GesturesHandPattern Recognition, AutomatedHumansNeural Networks, ComputerAlgorithmsDeep Learning

Stichwörter der Autor:innen

attention mechanismcontinuous hand gesture recognitiondeep learningfeature extractiongraph convolutional networkshand gesture recognitionhuman–computer interaction (HCI)skeleton basedtransformer

Autor:innen und Institutionen

  • Graduate School of Information, Production and System, Waseda University, Kitakyushu 808-0135, Japan.
  • Institute of Micromechanics and Photonics, Faculty of Mechatronics, Warsaw University of Technology, 00-661 Warszawa, Poland.
  • Waseda Institute of Advanced Study, Waseda University, Tokyo 169-8050, Japan.

ORCID-Profile: Pengcheng Han, Xin He, Takafumi Matsumaru, Vibekananda Dutta

Förderung

  • Waseda University Grant for Special Research Project
  • Japan Society for the Promotion of Science
  • JSPS KAKENHI

Zitieren

APA

Han, P., He, X., Matsumaru, T., & Dutta, V. (2025). Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition. Sensors. https://doi.org/10.3390/s25030702

Vancouver

Han P, He X, Matsumaru T, Dutta V. Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition. Sensors. 2025. doi:10.3390/s25030702

BibTeX
@article{Han2025,
  author = {Han, Pengcheng and He, Xin and Matsumaru, Takafumi and Dutta, Vibekananda},
  title = {Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition},
  journal = {Sensors},
  year = {2025},
  doi = {10.3390/s25030702}
}
Kostenlos starten →