Talk With Your Fingers: A Depth-Aware Benchmark for Air-Writing Recognition
IEEE Transactions on Circuits and Systems for Video Technology · 2026-07 · 36 (7) · 10632-10644
DOI: 10.1109/tcsvt.2026.3670256 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-07. Verfasst von Meiqi Wu und 7 weiteren Autor:innen. Die Arbeit nennt 79 Literaturangaben. Gefördert von: National Science and Technology Major Project, Strategic Priority Research Program of the Chinese Academy of Sciences, National Natural Science Foundation of China.
Abstract (Originalsprache)
Air-writing has emerged as a promising communication modality for AR/VR and metaverse environments, enabling quiet, non-contact text input by translating finger movements into natural language. However, existing approaches typically project in-air writing onto a virtual 2D plane and assume characters are formed with a single continuous stroke–an oversimplification that neglects the rich 3D structure inherent in natural handwriting. In this work, we challenge the “single-stroke 2D” paradigm and explore the role of depth cues in enhancing air-writing recognition. To this end, we present DAAWBench, the first large-scale Depth-Aware Air-Writing dataset, featuring 8.8 million RGB-D frames annotated with 3,755 Chinese characters from the GB2312-80 Level-1 set. Our analysis reveals consistent depth variations at stroke boundaries, indicating that stroke segmentation and character recognition can benefit from depth modeling. Based on these insights, we propose DARec, a novel 3D trajectory-based recognition model that effectively leverages depth-aware priors. Extensive experiments across in-domain and out-of-domain settings, including evaluations with vision-language models ( $e.g$ ., GPT-4o, Qwen-VL) and human baselines, show that DARec significantly outperforms 2D-only counterparts, achieving 87.73% accuracy versus 9.05%. Our findings demonstrate the critical importance of depth modeling in human-computer co-creative interfaces, and we will publicly release our dataset and code at https://github.com/wmeiqi/DAAWBench
Förderung
- National Science and Technology Major Project
- Strategic Priority Research Program of the Chinese Academy of Sciences
- National Natural Science Foundation of China
Zitieren
APA
Wu, M., Zhao, Y., Li, X., Hu, S., Cai, Y., Wu, J., Wang, W., & Huang, K. (2026). Talk With Your Fingers: A Depth-Aware Benchmark for Air-Writing Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 36(7), 10632-10644. https://doi.org/10.1109/tcsvt.2026.3670256
Vancouver
Wu M, Zhao Y, Li X, Hu S, Cai Y, Wu J, et al. Talk With Your Fingers: A Depth-Aware Benchmark for Air-Writing Recognition. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(7):10632-10644. doi:10.1109/tcsvt.2026.3670256
BibTeX
@article{Wu2026,
author = {Wu, Meiqi and Zhao, Yuzhong and Li, Xuchen and Hu, Shiyu and Cai, Yuanqiang and Wu, Jiahong and Wang, Weiqiang and Huang, Kaiqi},
title = {Talk With Your Fingers: A Depth-Aware Benchmark for Air-Writing Recognition},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {7},
pages = {10632--10644},
doi = {10.1109/tcsvt.2026.3670256}
}
EUVIMED