EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An Approach to Enable Human–3D Object Interaction Through Voice Commands in an Immersive Virtual Environment

Alessio Catalfamo, Antonio Celesti, Maria Fazio, A. F. M. Saifuddin Saif, Yu-Sheng Lin, Edelberto Franco Silva, Massimo Villari

Big Data and Cognitive Computing · 17.07.2025 · 9 (7) · 188

DOI: 10.3390/bdcc9070188 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 17.07.2025. Verfasst von Alessio Catalfamo und 6 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Italian Ministry of Health Piano Operativo Salute (POS) trajectory 2 “eHealth, diagnostica avanzata, medical device e mini invasività”, Italian PRIN 2022 project Tele-Rehabilitaiton as a Service (TRaaS).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Nowadays, the Metaverse is facing many challenges. In this context, Virtual Reality (VR) applications allowing voice-based human–3D object interactions are limited due to the current hardware/software limitations. In fact, adopting Automated Speech Recognition (ASR) systems to interact with 3D objects in VR applications through users’ voice commands presents significant challenges due to the hardware and software limitations of headset devices. This paper aims to bridge this gap by proposing a methodology to address these issues. In particular, starting from a Mel-Frequency Cepstral Coefficient (MFCC) extraction algorithm able to capture the unique characteristics of the user’s voice, we pass it as input to a Convolutional Neural Network (CNN) model. After that, in order to integrate the CNN model with a VR application running on a standalone headset, such as Oculus Quest, we converted it into an Open Neural Network Exchange (ONNX) format, i.e., a Machine Learning (ML) interoperability open standard format. The proposed system demonstrates good performance and represents a foundation for the development of user-centric, effective computing systems, enhancing accessibility to VR environments through voice-based commands. Experiments demonstrate that a native CNN model developed through TensorFlow presents comparable performances with respect to the corresponding CNN model converted into the ONNX format, paving the way towards the development of VR applications running in headsets controlled through the user’s voice.

Förderung

  • Italian Ministry of Health Piano Operativo Salute (POS) trajectory 2 “eHealth, diagnostica avanzata, medical device e mini invasività”
  • Italian PRIN 2022 project Tele-Rehabilitaiton as a Service (TRaaS)

Zitieren

APA

Catalfamo, A., Celesti, A., Fazio, M., Saif, A. F. M. S., Lin, Y., Silva, E. F., & Villari, M. (2025). An Approach to Enable Human–3D Object Interaction Through Voice Commands in an Immersive Virtual Environment. Big Data and Cognitive Computing, 9(7), 188. https://doi.org/10.3390/bdcc9070188

Vancouver

Catalfamo A, Celesti A, Fazio M, Saif AFMS, Lin Y, Silva EF, et al. An Approach to Enable Human–3D Object Interaction Through Voice Commands in an Immersive Virtual Environment. Big Data and Cognitive Computing. 2025;9(7):188. doi:10.3390/bdcc9070188

BibTeX
@article{Catalfamo2025,
  author = {Catalfamo, Alessio and Celesti, Antonio and Fazio, Maria and Saif, A. F. M. Saifuddin and Lin, Yu-Sheng and Silva, Edelberto Franco and Villari, Massimo},
  title = {An Approach to Enable Human–3D Object Interaction Through Voice Commands in an Immersive Virtual Environment},
  journal = {Big Data and Cognitive Computing},
  year = {2025},
  volume = {9},
  number = {7},
  pages = {188},
  doi = {10.3390/bdcc9070188}
}
Kostenlos starten →