EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Multi-Modal Beat Alignment Transformers for Dance Quality Assessment Framework

Taewan Kim

Journal of Multimedia Information System · 30.06.2024 · 11 (2) · 149-156

DOI: 10.33851/jmis.2024.11.2.149 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Multimedia Information System, online erschienen am 30.06.2024. Verfasst von Taewan Kim. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Dongduk Women`s University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, the dance entertainment industry has been growing as consumers have a desire to learn and improve their dancing skills. To fulfill this need, they need to be evaluated and get feedback to improve their dance skills, but the evaluation is very dependent on professional dancers. With the advent of deep learning techniques that can understand and learn the structure of 3D skeletons, graph convolutional networks and transformers have shown performance improvements in 3D human action understanding. In this paper, we propose Dance Quality Assessment (DanceQA) Framework to evaluate dance performance and predicts its dance quality numerically. For problem definition, we collect and capture 3D skeletal data by 3D pose estimator and label their dance quality. By analyzing the dataset, we propose dance quality measures, kinematic information entropy and multi-modal beat similarity, which consider traditional criteria for dance techniques. Based on results of the dance quality measures, kinematic entropy embedding matrix and multi-modal beat alignment transformers are designed to learns salient joints and frames in 3D dance sequence. Thus, we design the overall network architecture, DanceQA transformers, which consider spatial and temporal characteristics of 3D dance sequence from multiple input features and demonstrate that the proposed transformers outperform other Graph Convolutional Network (GCN)s and transformers on the DanceQA dataset. In numerous experiments, the CQTs outperforms previous methods, graph convolutional networks and multimodal transformers, at least by up to 0.146 in correlation coefficient.

Förderung

  • Dongduk Women`s University

Zitieren

APA

Kim, T. (2024). Multi-Modal Beat Alignment Transformers for Dance Quality Assessment Framework. Journal of Multimedia Information System, 11(2), 149-156. https://doi.org/10.33851/jmis.2024.11.2.149

Vancouver

Kim T. Multi-Modal Beat Alignment Transformers for Dance Quality Assessment Framework. Journal of Multimedia Information System. 2024;11(2):149-156. doi:10.33851/jmis.2024.11.2.149

BibTeX
@article{Kim2024,
  author = {Kim, Taewan},
  title = {Multi-Modal Beat Alignment Transformers for Dance Quality Assessment Framework},
  journal = {Journal of Multimedia Information System},
  year = {2024},
  volume = {11},
  number = {2},
  pages = {149--156},
  doi = {10.33851/jmis.2024.11.2.149}
}
Kostenlos starten →