EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Study of NLP-Based Speech Interfaces in Medical Virtual Reality

Mohit Nayak, Jari Kangas, Roope Raisamo

Multimodal Technologies and Interaction · 26.05.2025 · 9 (6) · 50

DOI: 10.3390/mti9060050 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Multimodal Technologies and Interaction, online erschienen am 26.05.2025. Verfasst von Mohit Nayak und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Council Of Finland, Business Finland.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Applications of virtual reality (VR) have grown in significance in medicine, as they are able to recreate real-life scenarios in 3D while posing reduced risks to patients. However, there are several interaction challenges to overcome when moving from 2D screens to 3D VR environments, such as complex controls and slow user adaptation. More intuitive techniques are needed for enhanced user experience. Our research explored the potential of intelligent speech interfaces to enhance user interaction while conducting complex medical tasks. We developed a speech-based assistant within a VR application for maxillofacial implant planning, leveraging natural language processing (NLP) to interpret user intentions and to execute tasks such as obtaining surgical equipment or answering questions related to the VR environment. The objective of the study was to evaluate the usability and cognitive load of the speech-based assistant. We conducted a mixed-methods within-subjects user study with 20 participants and compared the voice-assisted approach to traditional interaction methods, such as button panels on the VR view, across various tasks. Our findings indicate that NLP-driven speech-based assistants can enhance interaction and accessibility in medical VR, especially in areas such as locating controls, easiness of control, user comfort, and intuitive interaction. These findings highlight the potential benefits of augmenting traditional controls with speech interfaces, particularly in complex VR scenarios where conventional methods may limit usability. We identified key areas for future research, including improving the intelligence, accuracy, and user experience of speech-based systems. Addressing these areas could facilitate the development of more robust, user-centric, voice-assisted applications in virtual reality environments.

Förderung

  • Research Council Of Finland
  • Business Finland

Zitieren

APA

Nayak, M., Kangas, J., & Raisamo, R. (2025). A Study of NLP-Based Speech Interfaces in Medical Virtual Reality. Multimodal Technologies and Interaction, 9(6), 50. https://doi.org/10.3390/mti9060050

Vancouver

Nayak M, Kangas J, Raisamo R. A Study of NLP-Based Speech Interfaces in Medical Virtual Reality. Multimodal Technologies and Interaction. 2025;9(6):50. doi:10.3390/mti9060050

BibTeX
@article{Nayak2025,
  author = {Nayak, Mohit and Kangas, Jari and Raisamo, Roope},
  title = {A Study of NLP-Based Speech Interfaces in Medical Virtual Reality},
  journal = {Multimodal Technologies and Interaction},
  year = {2025},
  volume = {9},
  number = {6},
  pages = {50},
  doi = {10.3390/mti9060050}
}
Kostenlos starten →