Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition
Sensors · 2025
DOI: 10.3390/s25030702 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Pengcheng Han und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information, Production and System, Waseda University, Kitakyushu 808-0135, Japan. Hauptthemen laut MEDLINE: Gestures, Hand, Pattern Recognition, Automated. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Waseda University Grant for Special Research Project, Japan Society for the Promotion of Science, JSPS KAKENHI.
Abstract (Originalsprache)
Manually crafted features often suffer from being subjective, having an inadequate accuracy, or lacking in robustness in recognition. Meanwhile, existing deep learning methods often overlook the structural and dynamic characteristics of the human hand, failing to fully explore the contextual information of joints in both the spatial and temporal domains. To effectively capture dependencies between the hand joints that are not adjacent but may have potential connections, it is essential to learn long-term relationships. This study proposes a skeleton-based hand gesture recognition framework, the ST-KT, a spatio-temporal graph convolution network, and a transformer with the Kolmogorov–Arnold Network (KAN) model. It incorporates spatio-temporal graph convolution network (ST-GCN) modules and a spatio-temporal transformer module with KAN (KAN–Transformer). ST-GCN modules, which include a spatial graph convolution network (SGCN) and a temporal convolution network (TCN), extract primary features from skeleton sequences by leveraging the strength of graph convolutional networks in the spatio-temporal domain. A spatio-temporal position embedding method integrates node features, enriching representations by including node identities and temporal information. The transformer layer includes a spatial KAN–Transformer (S-KT) and a temporal KAN–Transformer (T-KT), which further extract joint features by learning edge weights and node embeddings, providing richer feature representations and the capability for nonlinear modeling. We evaluated the performance of our method on two challenging skeleton-based dynamic gesture datasets: our method achieved an accuracy of 97.5% on the SHREC’17 track dataset and 94.3% on the DHG-14/28 dataset. These results demonstrate that our proposed method, ST-KT, effectively captures dynamic skeleton changes and complex joint relationships.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Graduate School of Information, Production and System, Waseda University, Kitakyushu 808-0135, Japan.
- Institute of Micromechanics and Photonics, Faculty of Mechatronics, Warsaw University of Technology, 00-661 Warszawa, Poland.
- Waseda Institute of Advanced Study, Waseda University, Tokyo 169-8050, Japan.
ORCID-Profile: Pengcheng Han, Xin He, Takafumi Matsumaru, Vibekananda Dutta
Förderung
- Waseda University Grant for Special Research Project
- Japan Society for the Promotion of Science
- JSPS KAKENHI
Zitieren
APA
Han, P., He, X., Matsumaru, T., & Dutta, V. (2025). Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition. Sensors. https://doi.org/10.3390/s25030702
Vancouver
Han P, He X, Matsumaru T, Dutta V. Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition. Sensors. 2025. doi:10.3390/s25030702
BibTeX
@article{Han2025,
author = {Han, Pengcheng and He, Xin and Matsumaru, Takafumi and Dutta, Vibekananda},
title = {Spatio-Temporal Transformer with Kolmogorov–Arnold Network for Skeleton-Based Hand Gesture Recognition},
journal = {Sensors},
year = {2025},
doi = {10.3390/s25030702}
}
EUVIMED