EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Real-Time Dual-Modal 6D Pose Estimation for Virtual Reality Interaction Using Deep Learning

Rong Hu, Runlin Li, Ping Huang

INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL · 03.11.2026 · 21 (6)

DOI: 10.15837/ijccc.2026.6.7424 ↗

Kurz gefasst

Fachartikel in der Zeitschrift INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, online erschienen am 03.11.2026. Verfasst von Rong Hu und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate and low-latency 6D pose estimation is a key requirement for natural interaction in virtual reality systems, yet existing methods often struggle to simultaneously satisfy precision, real-time performance, and robustness in multi-user or interference-prone environments. To address these challenges, this study proposes a deep learning-based dual-modal 6D pose estimation framework that integrates RGB and depth sensing for real-time virtual reality interaction. The proposed method employs a two-stream feature extraction architecture, where two-dimensional texture features are extracted from RGB images using a ResNet-18 backbone enhanced with efficient channel attention, while three-dimensional geometric features are obtained from depth data through an SO(3)-equivariant encoder combined with vector neuron linear layers and multilayer perceptrons. A cross-modal fusion strategy is introduced to achieve complementary enhancement between texture and geometric representations, followed by pose regression based on point pair feature matching. Experimental results on public benchmarks and multi-user virtual interaction scenarios demonstrate that the proposed approach achieves a median rotation error of 0.42◦ and a translation error of 0.38 cm, representing an error reduction of over 67% compared to models without SO(3) encoding. The system maintains a success rate of 95-98% under normal illumination and achieves a processing latency of 4.81 ms with 208 FPS on high-end hardware. Furthermore, the framework exhibits strong robustness to lighting variation, occlusion, and cross-user interference. These results indicate that the proposed dual-modal, SO(3)-aware pose estimation framework provides an effective computational solution for real-time 6D pose tracking and control in virtual reality interaction systems.

Zitieren

APA

Hu, R., Li, R., & Huang, P. (2026). Real-Time Dual-Modal 6D Pose Estimation for Virtual Reality Interaction Using Deep Learning. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, 21(6). https://doi.org/10.15837/ijccc.2026.6.7424

Vancouver

Hu R, Li R, Huang P. Real-Time Dual-Modal 6D Pose Estimation for Virtual Reality Interaction Using Deep Learning. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL. 2026;21(6). doi:10.15837/ijccc.2026.6.7424

BibTeX
@article{Hu2026,
  author = {Hu, Rong and Li, Runlin and Huang, Ping},
  title = {Real-Time Dual-Modal 6D Pose Estimation for Virtual Reality Interaction Using Deep Learning},
  journal = {INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL},
  year = {2026},
  volume = {21},
  number = {6},
  doi = {10.15837/ijccc.2026.6.7424}
}
Kostenlos starten →