EUVIMEDScopri EUVIMED
Video/Audio

A Gesture- and Voice-controlled Virtual Reality System for Immersive Home Design: System Design and User Experience Analysis

Siyuan Zhao, Xiangkun Liu

Journal of Visualized Experiments · 2026

DOI: 10.3791/70051 ↗

At a glance

Video/Audio in Journal of Visualized Experiments, 2026. Written by Siyuan Zhao and 1 co-authors. Institution: School of Architecture, Building & Design, Faculty of Innovation & Technology, Taylor's University Lakeside Campus. Major MEDLINE topics: Gestures, Virtual Reality, User-Computer Interface, Voice. 44 references.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Gesture- and voice-based interaction in VR has shown promise in various applications; however, existing systems are hindered by their emphasis on navigation tasks, dependence on two-handed gestures, lack of fine-grained precision, and testing on small, homogeneous populations. These limitations limit applicability to intricate, design-centric workflows. To address these limitations, we introduce a Gesture- and Voice-Controlled Virtual Reality System for immersive home design, which incorporates the multimodal fusion of hand-tracked gesture input and natural language instructions for precise furniture placement, material selection, and spatial adjustment. Three aspects of novelty in the system are (i) adaptive modes of interaction to support one-handed usage, numeric voice commands, and coarse-to-fine adjustment of precision; (ii) multimodal fusion architecture merging gesture, speech, and context information for sub-centimeter precision; and (iii) task-aware user experience evaluation framework tailored to home design processes. The development process follows a systematic pipeline, from requirements analysis and design of gesture and voice lexicons, to multimodal intent recognition, VR scene assembly with snapping and alignment, and iterative testing. Experimental results on a wide variety of participant populations demonstrate a 30% decrease in placement error and substantial improvements in usability and workload ratings over controller-based interaction. The results suggest the potential of multimodal VR for creating accessible and engaging home design experiences.

Subjects

MeSH terms (MEDLINE)

GesturesVirtual RealityUser-Computer InterfaceVoiceHumans

Authors and institutions

  • School of Architecture, Building & Design, Faculty of Innovation & Technology, Taylor's University Lakeside Campus.
  • The Design School, Faculty of Innovation & Technology, Taylor's University Lakeside Campus; liuxiangkun@ldy.edu.rs.

Cite

APA

Zhao, S., & Liu, X. (2026). A Gesture- and Voice-controlled Virtual Reality System for Immersive Home Design: System Design and User Experience Analysis. Journal of Visualized Experiments. https://doi.org/10.3791/70051

Vancouver

Zhao S, Liu X. A Gesture- and Voice-controlled Virtual Reality System for Immersive Home Design: System Design and User Experience Analysis. Journal of Visualized Experiments. 2026. doi:10.3791/70051

BibTeX
@article{Zhao2026,
  author = {Zhao, Siyuan and Liu, Xiangkun},
  title = {A Gesture- and Voice-controlled Virtual Reality System for Immersive Home Design: System Design and User Experience Analysis},
  journal = {Journal of Visualized Experiments},
  year = {2026},
  doi = {10.3791/70051}
}
Start for free →