QEAN: quaternion-enhanced attention network for visual dance generation
The Visual Computer · 2025-01 · 41 (2) · 961-973
DOI: 10.1007/s00371-024-03376-5 ↗Kurz gefasst
Fachartikel in der Zeitschrift The Visual Computer, online erschienen am 14.04.2024. Verfasst von Zhizhen Zhou und 6 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 42 Literaturangaben. Gefördert von: Key Areas Research and Development Program of Guangzhou, Science and technology research in key areas in Foshan, Special Project for Research and Development in Key areas of Guangdong Province.
Abstract (Originalsprache)
The study of music-generated dance is a novel and challenging image generation task. It aims to input a piece of music and seed motions, then generate natural dance movements for the subsequent music. Transformer-based methods face challenges in time-series prediction tasks related to human movements and music due to their struggle in capturing the nonlinear relationship and temporal aspects. This can lead to issues like joint deformation, role deviation, floating, and inconsistencies in dance movements generated in response to the music. In this paper, we propose a quaternion-enhanced attention network for visual dance synthesis from a quaternion perspective, which consists of a spin position embedding (SPE) module and a quaternion rotary attention (QRA) module. First, SPE embeds position information into self-attention in a rotational manner, leading to better learning of features of movement sequences and audio sequences and improved understanding of the connection between music and dance. Second, QRA represents and fuses 3D motion features and audio features in the form of a series of quaternions, enabling the model to better learn the temporal coordination of music and dance under the complex temporal cycle conditions of dance generation. Finally, we conducted experiments on the dataset AIST++, and the results show that our approach achieves better and more robust performance in generating accurate, high-quality dance movements. Our source code and dataset can be available from https://github.com/MarasyZZ/QEAN and https://google.github.io/aistplusplus_dataset, respectively.
Förderung
- Key Areas Research and Development Program of Guangzhou
- Science and technology research in key areas in Foshan
- Special Project for Research and Development in Key areas of Guangdong Province
- Science and technology projects of Guangzhou
- Guangdong Provincial Key Laboratory of Cyber-Physical System
- National Natural Science Foundation
- Science and Technology Planning Project of Guangdong Province
- Science and Technology Projects in Guangzhou
Zitieren
APA
Zhou, Z., Huo, Y., Huang, G., Zeng, A., Chen, X., Huang, L., & Li, Z. (2025). QEAN: quaternion-enhanced attention network for visual dance generation. The Visual Computer, 41(2), 961-973. https://doi.org/10.1007/s00371-024-03376-5
Vancouver
Zhou Z, Huo Y, Huang G, Zeng A, Chen X, Huang L, et al. QEAN: quaternion-enhanced attention network for visual dance generation. The Visual Computer. 2025;41(2):961-973. doi:10.1007/s00371-024-03376-5
BibTeX
@article{Zhou2025,
author = {Zhou, Zhizhen and Huo, Yejing and Huang, Guoheng and Zeng, An and Chen, Xuhang and Huang, Lian and Li, Zinuo},
title = {QEAN: quaternion-enhanced attention network for visual dance generation},
journal = {The Visual Computer},
year = {2025},
volume = {41},
number = {2},
pages = {961--973},
doi = {10.1007/s00371-024-03376-5}
}
EUVIMED