EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Dance Motion-Guided Music Generation via Residual Vector Quantization

Shuhong Lin, Moshe Zukerman, Hong Yan

Electronics · 14.05.2026 · 15 (10) · 2098

DOI: 10.3390/electronics15102098 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 14.05.2026. Verfasst von Shuhong Lin und 2 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Hong Kong Innovation and Technology Commission, City University of Hong Kong.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Music is a traditional and important element in human entertainment. We propose a novel deep learning-based method for generating music from human dance movements. We employ Residual Vector Quantization (RVQ) for music tokenization, using RVQ indices as the music representation, thereby reducing learning complexity. A cross-modal generation model integrating LSTM and attention mechanisms is designed to generate RVQ codebook indices from dance motions. Finally, the indices can be converted back to music waveforms through a music decoder. Experimental results demonstrate the feasibility of generating coherent music that aligns with dance dynamics, providing a new approach for cross-modal entertainment content creation.

Förderung

  • Hong Kong Innovation and Technology Commission
  • City University of Hong Kong

Zitieren

APA

Lin, S., Zukerman, M., & Yan, H. (2026). Dance Motion-Guided Music Generation via Residual Vector Quantization. Electronics, 15(10), 2098. https://doi.org/10.3390/electronics15102098

Vancouver

Lin S, Zukerman M, Yan H. Dance Motion-Guided Music Generation via Residual Vector Quantization. Electronics. 2026;15(10):2098. doi:10.3390/electronics15102098

BibTeX
@article{Lin2026,
  author = {Lin, Shuhong and Zukerman, Moshe and Yan, Hong},
  title = {Dance Motion-Guided Music Generation via Residual Vector Quantization},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {10},
  pages = {2098},
  doi = {10.3390/electronics15102098}
}
Kostenlos starten →