EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

TSMTFN: Two-Stream Temporal Shift Module Network for Efficient Egocentric Gesture Recognition in Virtual Reality

Muhammad Abrar Hussain, Chanjun Chun, SeongKi Kim

Virtual Worlds · 04.12.2025 · 4 (4) · 58

DOI: 10.3390/virtualworlds4040058 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Virtual Worlds, online erschienen am 04.12.2025. Verfasst von Muhammad Abrar Hussain und 2 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Regional Innovation System & Education (RISE) program through the (Gwangju RISE Center), the Ministry of Education (MOE) and the (Gwangju Metropolitan City), R…, Korean government.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Egocentric hand gesture recognition is vital for natural human–computer interaction in augmented and virtual reality (AR/VR) systems. However, most deep learning models struggle to balance accuracy and efficiency, limiting real-time use on wearable devices. This paper introduces a Two-Stream Temporal Shift Module Transformer Fusion Network (TSMTFN) that achieves high recognition accuracy with low computational cost. The model integrates Temporal Shift Modules (TSMs) for efficient motion modeling and a Transformer-based fusion mechanism for long-range temporal understanding, operating on dual RGB-D streams to capture complementary visual and depth cues. Training stability and generalization are enhanced through full-layer training from epoch 1 and MixUp/CutMix augmentations. Evaluated on the EgoGesture dataset, TSMTFN attained 96.18% top-1 accuracy and 99.61% top-5 accuracy on the independent test set with only 16 GFLOPs and 21.3M parameters, offering a 2.4–4.7× reduction in computation compared to recent state-of-the-art methods. The model runs at 15.10 samples/s, achieving real-time performance. The results demonstrate robust recognition across over 95% of gesture classes and minimal inter-class confusion, establishing TSMTFN as an efficient, accurate, and deployable solution for next-generation wearable AR/VR gesture interfaces.

Förderung

  • Regional Innovation System & Education (RISE) program through the (Gwangju RISE Center), the Ministry of Education (MOE) and the (Gwangju Metropolitan City), R…
  • Korean government

Zitieren

APA

Hussain, M. A., Chun, C., & Kim, S. (2025). TSMTFN: Two-Stream Temporal Shift Module Network for Efficient Egocentric Gesture Recognition in Virtual Reality. Virtual Worlds, 4(4), 58. https://doi.org/10.3390/virtualworlds4040058

Vancouver

Hussain MA, Chun C, Kim S. TSMTFN: Two-Stream Temporal Shift Module Network for Efficient Egocentric Gesture Recognition in Virtual Reality. Virtual Worlds. 2025;4(4):58. doi:10.3390/virtualworlds4040058

BibTeX
@article{Hussain2025,
  author = {Hussain, Muhammad Abrar and Chun, Chanjun and Kim, SeongKi},
  title = {TSMTFN: Two-Stream Temporal Shift Module Network for Efficient Egocentric Gesture Recognition in Virtual Reality},
  journal = {Virtual Worlds},
  year = {2025},
  volume = {4},
  number = {4},
  pages = {58},
  doi = {10.3390/virtualworlds4040058}
}
Kostenlos starten →