EUVIMEDEUVIMED entdecken
Crossref

Gesture Builder: Flexible Gesture Customization and Efficient Recognition on VR Devices

Yang Zou, Yanguang Wan, Yonghao Zhang, Xiao Zhou, Jian Cheng, Cuixia Ma, Chun Yu, Xiaoming Deng, Hongan Wang

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 09.06.2025 · 9 (2) · 1-37

DOI: 10.1145/3729484 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 18.06.2025. Verfasst von Yang Zou und 8 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 83 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Beijing Natural Science Foundation, National Science and Technology Major Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the realm of VR/MR interactions, gestures serve as a critical bridge between users and interfaces, with custom gestures enhancing creativity and providing a personalized immersive experience. We introduce a novel gesture definition and recognition framework that allows users to customize a wide array of gestures by demonstrating them just three times. A major challenge lies in effectively representing gestures computationally. To address this, we have pre-trained a hand posture representation model using a Vector Quantized Variational Autoencoder (VQ-VAE) with a codebook of adaptive size, allowing hand postures defined by 23 joint positions of the hand to be projected into a latent space. In this space, different postures are formed into clusters, and a testing posture can be assigned to a cluster by a specific distance metric. The dynamic gestures are then represented as sequences of discrete hand postures and wrist positions. Employing a straightforward sequence matching algorithm, our framework achieves highly efficient recognition with minimal computational demands. We evaluated this system through a user study that includes 16 pre-defined gestures and 106 user-defined gestures. The results confirm that our system can provide robust real-time gesture recognition and effectively supports the customization of gestures according to user preferences. Our approach surpasses previous methods by enhancing gesture diversity and reducing constraints on gesture customization. Project page: https://iscas3dv.github.io/GestureBuilder/.

Förderung

  • National Natural Science Foundation of China
  • Beijing Natural Science Foundation
  • National Science and Technology Major Project

Zitieren

APA

Zou, Y., Wan, Y., Zhang, Y., Zhou, X., Cheng, J., Ma, C., Yu, C., Deng, X., & Wang, H. (2025). Gesture Builder: Flexible Gesture Customization and Efficient Recognition on VR Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 9(2), 1-37. https://doi.org/10.1145/3729484

Vancouver

Zou Y, Wan Y, Zhang Y, Zhou X, Cheng J, Ma C, et al. Gesture Builder: Flexible Gesture Customization and Efficient Recognition on VR Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2025;9(2):1-37. doi:10.1145/3729484

BibTeX
@article{Zou2025,
  author = {Zou, Yang and Wan, Yanguang and Zhang, Yonghao and Zhou, Xiao and Cheng, Jian and Ma, Cuixia and Yu, Chun and Deng, Xiaoming and Wang, Hongan},
  title = {Gesture Builder: Flexible Gesture Customization and Efficient Recognition on VR Devices},
  journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  year = {2025},
  volume = {9},
  number = {2},
  pages = {1--37},
  doi = {10.1145/3729484}
}
Kostenlos starten →