EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

DExter: Learning and Controlling Performance Expression with Diffusion Models

Huan Zhang, Shreyan Chowdhury, Carlos Eduardo Cancino-Chacón, Jinhua Liang, Simon Dixon, Gerhard Widmer

Applied Sciences · 26.07.2024 · 14 (15) · 6543

DOI: 10.3390/app14156543 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 26.07.2024. Verfasst von Huan Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: UKRI Centre for Doctoral Training in Artificial Intelligence and Music, UK Research and Innovation, European Research Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the pursuit of developing expressive music performance models using artificial intelligence, this paper introduces DExter, a new approach leveraging diffusion probabilistic models to render Western classical piano performances. The main challenge faced in performance rendering tasks is the continuous and sequential modeling of expressive timing and dynamics over time, which is critical for capturing the evolving nuances that characterize live musical performances. In this approach, performance parameters are represented in a continuous expression space, and a diffusion model is trained to predict these continuous parameters while being conditioned on a musical score. Furthermore, DExter also enables the generation of interpretations (expressive variations of a performance) guided by perceptually meaningful features by being jointly conditioned on score and perceptual-feature representations. Consequently, we find that our model is useful for learning expressive performance, generating perceptually steered performances, and transferring performance styles. We assess the model through quantitative and qualitative analyses, focusing on specific performance metrics regarding dimensions like asynchrony and articulation, as well as through listening tests that compare generated performances with different human interpretations. The results show that DExter is able to capture the time-varying correlation of the expressive parameters, and it compares well to existing rendering models in subjectively evaluated ratings. The perceptual-feature-conditioned generation and transferring capabilities of DExter are verified via a proxy model predicting perceptual characteristics of differently steered performances.

Förderung

  • UKRI Centre for Doctoral Training in Artificial Intelligence and Music
  • UK Research and Innovation
  • European Research Council

Zitieren

APA

Zhang, H., Chowdhury, S., Cancino-Chacón, C. E., Liang, J., Dixon, S., & Widmer, G. (2024). DExter: Learning and Controlling Performance Expression with Diffusion Models. Applied Sciences, 14(15), 6543. https://doi.org/10.3390/app14156543

Vancouver

Zhang H, Chowdhury S, Cancino-Chacón CE, Liang J, Dixon S, Widmer G. DExter: Learning and Controlling Performance Expression with Diffusion Models. Applied Sciences. 2024;14(15):6543. doi:10.3390/app14156543

BibTeX
@article{Zhang2024,
  author = {Zhang, Huan and Chowdhury, Shreyan and Cancino-Chacón, Carlos Eduardo and Liang, Jinhua and Dixon, Simon and Widmer, Gerhard},
  title = {DExter: Learning and Controlling Performance Expression with Diffusion Models},
  journal = {Applied Sciences},
  year = {2024},
  volume = {14},
  number = {15},
  pages = {6543},
  doi = {10.3390/app14156543}
}
Kostenlos starten →