Towards Smartphone-based 3D Hand Pose Reconstruction Using Acoustic Signals
ACM Transactions on Sensor Networks · 30.09.2024 · 20 (5) · 1-32
DOI: 10.1145/3677122 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Sensor Networks, online erschienen am 26.08.2024. Verfasst von Shiyang Wang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 96 Literaturangaben.
Abstract (Originalsprache)
Accurately reconstructing 3D hand poses is a pivotal element for numerous Human-Computer Interaction applications. In this work, we propose SonicHand, the first smartphone-based 3D hand pose reconstruction system using purely inaudible acoustic signals. SonicHand incorporates signal processing techniques and a deep learning framework to address a series of challenges. First, it encodes the topological information of the hand skeleton as prior knowledge and utilizes a deep learning model to realistically and smoothly reconstruct the hand poses. Second, the system employs adversarial training to enhance the generalization ability of our system to be deployed in a new environment or for a new user. Third, we adopt a hand tracking method based on channel impulse response estimation. It enables our system to handle the scenario where the hand performs gestures while moving arbitrarily as a whole. We conduct extensive experiments on a smartphone testbed to demonstrate the effectiveness and robustness of our system from various dimensions. The experiments involve 10 subjects performing up to 12 different hand gestures in three distinctive environments. When the phone is held in one of the user’s hands, the proposed system can track joints with an average error of 18.64 mm.
Zitieren
APA
Wang, S., Wang, X., Jiang, W., Miao, C., Cao, Q., Wang, H., Sun, K., Xue, H., & Su, L. (2024). Towards Smartphone-based 3D Hand Pose Reconstruction Using Acoustic Signals. ACM Transactions on Sensor Networks, 20(5), 1-32. https://doi.org/10.1145/3677122
Vancouver
Wang S, Wang X, Jiang W, Miao C, Cao Q, Wang H, et al. Towards Smartphone-based 3D Hand Pose Reconstruction Using Acoustic Signals. ACM Transactions on Sensor Networks. 2024;20(5):1-32. doi:10.1145/3677122
BibTeX
@article{Wang2024,
author = {Wang, Shiyang and Wang, Xingchen and Jiang, Wenjun and Miao, Chenglin and Cao, Qiming and Wang, Haoyu and Sun, Ke and Xue, Hongfei and Su, Lu},
title = {Towards Smartphone-based 3D Hand Pose Reconstruction Using Acoustic Signals},
journal = {ACM Transactions on Sensor Networks},
year = {2024},
volume = {20},
number = {5},
pages = {1--32},
doi = {10.1145/3677122}
}
EUVIMED