An Augmented Reality Programming Approach to Incorporate Spatial Multimodal Interaction
Journal of Computing and Information Science in Engineering · 01.09.2025 · 25 (9)
DOI: 10.1115/1.4069117 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Computing and Information Science in Engineering, online erschienen am 25.07.2025. Verfasst von Jingquan Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 36 Literaturangaben. Gefördert von: Shanxi Scholarship Council of China.
Abstract (Originalsprache)
With the wide application of mobile robots, traditional robot programming faces a series of challenges due to its complex task settings and user interactions. In the context of Industry 5.0, we are committed to researching a spatial programming method based on AR (augmented reality) technology to realize human–robot collaboration. In this article, the AR-based spatial perception method is studied, and the virtual coordinate system in AR is established through the virtual reality model fusion perception method. By analyzing the relationship between each virtual coordinate system, the robot virtual space operation is planned, and then the physical world robot operation is controlled. A spatial multimodal AR interaction interface covering gesture, speech, EEG signals, and gaze control is designed to realize different functions for each modal interaction method. In the laboratory environment, the feasibility and effectiveness of the system are tested and verified by taking the coal mining machine interceptor tooth repair task as an example and selecting multiple coal mining machine interceptor tooth repairs for flexible operation tasks in the testing process. The results of the study show that the spatial programming method based on AR technology can significantly improve the efficiency of task execution of operator-controlled mobile robots and enhance the human–robot interaction experience.
Förderung
- Shanxi Scholarship Council of China
Zitieren
APA
Liu, J., Wang, X., Xie, J., Liu, S., Qin, L., & Shen, W. (2025). An Augmented Reality Programming Approach to Incorporate Spatial Multimodal Interaction. Journal of Computing and Information Science in Engineering, 25(9). https://doi.org/10.1115/1.4069117
Vancouver
Liu J, Wang X, Xie J, Liu S, Qin L, Shen W. An Augmented Reality Programming Approach to Incorporate Spatial Multimodal Interaction. Journal of Computing and Information Science in Engineering. 2025;25(9). doi:10.1115/1.4069117
BibTeX
@article{Liu2025,
author = {Liu, Jingquan and Wang, Xuewen and Xie, Jiacheng and Liu, Shuguang and Qin, Lang and Shen, Weidong},
title = {An Augmented Reality Programming Approach to Incorporate Spatial Multimodal Interaction},
journal = {Journal of Computing and Information Science in Engineering},
year = {2025},
volume = {25},
number = {9},
doi = {10.1115/1.4069117}
}
EUVIMED