“I Can See Your Password”: A Case Study About Cybersecurity Risks in Mid-Air Interactions of Mixed Reality-Based Smart Manufacturing Applications
Journal of Computing and Information Science in Engineering · 01.03.2024 · 24 (3)
DOI: 10.1115/1.4062658 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Computing and Information Science in Engineering, online erschienen am 10.10.2023. Verfasst von Wenhao Yang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 60 Literaturangaben. Gefördert von: Division of Graduate Education, Division of Information and Intelligent Systems, Facebook.
Abstract (Originalsprache)
This paper aims to present a potential cybersecurity risk existing in mixed reality (MR)-based smart manufacturing applications that decipher digital passwords through a single RGB camera to capture the user’s mid-air gestures. We first created a test bed, which is an MR-based smart factory management system consisting of mid-air gesture-based user interfaces (UIs) on a video see-through MR head-mounted display. To interact with UIs and input information, the user’s hand movements and gestures are tracked by the MR system. We setup the experiment to be the estimation of the password input by users through mid-air hand gestures on a virtual numeric keypad. To achieve this goal, we developed a lightweight machine learning-based hand position tracking and gesture recognition method. This method takes either video streaming or recorded video clips (taken by a single RGB camera in front of the user) as input, where the videos record the users’ hand movements and gestures but not the virtual UIs. With the assumption of the known size, position, and layout of the keypad, the machine learning method estimates the password through hand gesture recognition and finger position detection. The evaluation result indicates the effectiveness of the proposed method, with a high accuracy of 97.03%, 94.06%, and 83.83% for 2-digit, 4-digit, and 6-digit passwords, respectively, using real-time video streaming as input with known length condition. Under the unknown length condition, the proposed method reaches 85.50%, 76.15%, and 77.89% accuracy for 2-digit, 4-digit, and 6-digit passwords, respectively.
Förderung
- Division of Graduate Education
- Division of Information and Intelligent Systems
Zitieren
APA
Yang, W., Dengxiong, X., Wang, X., Hu, Y., & Zhang, Y. (2024). “I Can See Your Password”: A Case Study About Cybersecurity Risks in Mid-Air Interactions of Mixed Reality-Based Smart Manufacturing Applications. Journal of Computing and Information Science in Engineering, 24(3). https://doi.org/10.1115/1.4062658
Vancouver
Yang W, Dengxiong X, Wang X, Hu Y, Zhang Y. “I Can See Your Password”: A Case Study About Cybersecurity Risks in Mid-Air Interactions of Mixed Reality-Based Smart Manufacturing Applications. Journal of Computing and Information Science in Engineering. 2024;24(3). doi:10.1115/1.4062658
BibTeX
@article{Yang2024,
author = {Yang, Wenhao and Dengxiong, Xiwen and Wang, Xueting and Hu, Yidan and Zhang, Yunbo},
title = {“I Can See Your Password”: A Case Study About Cybersecurity Risks in Mid-Air Interactions of Mixed Reality-Based Smart Manufacturing Applications},
journal = {Journal of Computing and Information Science in Engineering},
year = {2024},
volume = {24},
number = {3},
doi = {10.1115/1.4062658}
}
EUVIMED