EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition

Zengzhao Chen, Wenkai Huang, Hai Liu, Zhuo Wang, Yuqun Wen, Shengming Wang

Sensors · 18.04.2024 · 24 (8) · 2589

DOI: 10.3390/s24082589 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 18.04.2024. Verfasst von Zengzhao Chen und 5 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430079, China. Hauptthemen laut MEDLINE: Gestures. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The National Natural Science Foundation of China, The Research Project of National Collaborative Innovation Experimental Base for Teacher Development of Central China Normal University, National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Teaching gesture recognition is a technique used to recognize the hand movements of teachers in classroom teaching scenarios. This technology is widely used in education, including for classroom teaching evaluation, enhancing online teaching, and assisting special education. However, current research on gesture recognition in teaching mainly focuses on detecting the static gestures of individual students and analyzing their classroom behavior. To analyze the teacher’s gestures and mitigate the difficulty of single-target dynamic gesture recognition in multi-person teaching scenarios, this paper proposes skeleton-based teaching gesture recognition (ST-TGR), which learns through spatio-temporal representation. This method mainly uses the human pose estimation technique RTMPose to extract the coordinates of the keypoints of the teacher’s skeleton and then inputs the recognized sequence of the teacher’s skeleton into the MoGRU action recognition network for classifying gesture actions. The MoGRU action recognition module mainly learns the spatio-temporal representation of target actions by stacking a multi-scale bidirectional gated recurrent unit (BiGRU) and using improved attention mechanism modules. To validate the generalization of the action recognition network model, we conducted comparative experiments on datasets including NTU RGB+D 60, UT-Kinect Action3D, SBU Kinect Interaction, and Florence 3D. The results indicate that, compared with most existing baseline models, the model proposed in this article exhibits better performance in recognition accuracy and speed.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

GesturesHumansPattern Recognition, AutomatedAlgorithmsTeaching

Stichwörter der Autor:innen

action recognitionclassroom scenariopose estimationteaching gesture

Autor:innen und Institutionen

  • Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430079, China.
  • National Engineering Research Center for E-Learning, Central China Normal University, Wuhan 430079, China.
  • Faculty of Literature and Journalism, Xiangtan University, Xiangtan 411105, China.
  • National Engineering Research Center of Big Data, Center China Normal University, Wuhan 430079, China.

Förderung

  • The National Natural Science Foundation of China
  • The Research Project of National Collaborative Innovation Experimental Base for Teacher Development of Central China Normal University
  • National Natural Science Foundation of China
  • Research Project of National Collaborative Innovation Experimental Base for Teacher Development of Central China Normal University
  • Jiangxi Provincial Natural Science Foundation
  • National Natural Science Foundation of Hubei Province
  • university teaching reform research project of Jiangxi Province

Zitieren

APA

Chen, Z., Huang, W., Liu, H., Wang, Z., Wen, Y., & Wang, S. (2024). ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition. Sensors, 24(8), 2589. https://doi.org/10.3390/s24082589

Vancouver

Chen Z, Huang W, Liu H, Wang Z, Wen Y, Wang S. ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition. Sensors. 2024;24(8):2589. doi:10.3390/s24082589

BibTeX
@article{Chen2024,
  author = {Chen, Zengzhao and Huang, Wenkai and Liu, Hai and Wang, Zhuo and Wen, Yuqun and Wang, Shengming},
  title = {ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {8},
  pages = {2589},
  doi = {10.3390/s24082589}
}
Kostenlos starten →