EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Design and research of music teaching system based on virtual reality system in the context of education informatization

Yan Feng iD

PLOS ONE · 05.10.2023 · 18 (10) · e0285331

DOI: 10.1371/journal.pone.0285331 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 05.10.2023. Verfasst von Yan Feng. Beteiligte Einrichtung: School of Medical Humanities, Xinxiang Medical University, Xinxiang, 453000, China. Hauptthemen laut MEDLINE: Music, Virtual Reality. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Construction of public music curriculum system in colleges and universities based on red culture.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Virtual Reality (VR) technology uses computers to simulate the real world comprehensively. VR has been widely used in college teaching and has a huge application prospect. To better apply computer-aided instruction technology in music teaching, a music teaching system based on VR technology is proposed. First, a virtual piano is developed using the HTC Vive kit and the Leap Motion sensor fixed on the helmet as the hardware platform, and using Unity3D, related SteamVR plug-ins, and Leap Motion plug-ins as software platforms. Then, a gesture recognition algorithm is proposed and implemented. Specifically, the Dual Channel Convolutional Neural Network (DCCNN) is adopted to collect the user’s gesture command data. The dual-size convolution kernel is applied to extract the feature information in the image and the gesture command in the video, and then the DCCNN recognizes it. After the spatial and temporal information is extracted, Red-Green-Blue (RGB) color pattern images and optical flow images are input into the DCCNN. The prediction results are merged to obtain the final recognition result. The experimental results reveal that the recognition accuracy of DCCNN for the Curwen gesture is as high as 96%, and the recognition accuracy varies with different convolution kernels. By comparison, it is found that the recognition effect of DCCNN is affected by the size of the convolution kernel. Combining convolution kernels of size 5×5 and 7×7 can improve the recognition accuracy to 98%. The research results of this study can be used for music teaching piano and other VR products, with extensive popularization and application value.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

MusicVirtual RealityHumansNeural Networks, ComputerAlgorithmsUser-Computer Interface

Autor:innen und Institutionen

  • School of Medical Humanities, Xinxiang Medical University, Xinxiang, 453000, China.

ORCID-Profile: Yan Feng

Förderung

  • Construction of public music curriculum system in colleges and universities based on red culture

Zitieren

APA

Feng, Y. (2023). Design and research of music teaching system based on virtual reality system in the context of education informatization. PLOS ONE, 18(10), e0285331. https://doi.org/10.1371/journal.pone.0285331

Vancouver

Feng Y. Design and research of music teaching system based on virtual reality system in the context of education informatization. PLOS ONE. 2023;18(10):e0285331. doi:10.1371/journal.pone.0285331

BibTeX
@article{Feng2023,
  author = {Feng, Yan},
  title = {Design and research of music teaching system based on virtual reality system in the context of education informatization},
  journal = {PLOS ONE},
  year = {2023},
  volume = {18},
  number = {10},
  pages = {e0285331},
  doi = {10.1371/journal.pone.0285331}
}
Kostenlos starten →