DExter: Learning and Controlling Performance Expression with Diffusion Models
Applied Sciences · 26.07.2024 · 14 (15) · 6543
DOI: 10.3390/app14156543 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 26.07.2024. Verfasst von Huan Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: UKRI Centre for Doctoral Training in Artificial Intelligence and Music, UK Research and Innovation, European Research Council.
Abstract (Originalsprache)
In the pursuit of developing expressive music performance models using artificial intelligence, this paper introduces DExter, a new approach leveraging diffusion probabilistic models to render Western classical piano performances. The main challenge faced in performance rendering tasks is the continuous and sequential modeling of expressive timing and dynamics over time, which is critical for capturing the evolving nuances that characterize live musical performances. In this approach, performance parameters are represented in a continuous expression space, and a diffusion model is trained to predict these continuous parameters while being conditioned on a musical score. Furthermore, DExter also enables the generation of interpretations (expressive variations of a performance) guided by perceptually meaningful features by being jointly conditioned on score and perceptual-feature representations. Consequently, we find that our model is useful for learning expressive performance, generating perceptually steered performances, and transferring performance styles. We assess the model through quantitative and qualitative analyses, focusing on specific performance metrics regarding dimensions like asynchrony and articulation, as well as through listening tests that compare generated performances with different human interpretations. The results show that DExter is able to capture the time-varying correlation of the expressive parameters, and it compares well to existing rendering models in subjectively evaluated ratings. The perceptual-feature-conditioned generation and transferring capabilities of DExter are verified via a proxy model predicting perceptual characteristics of differently steered performances.
Förderung
- UKRI Centre for Doctoral Training in Artificial Intelligence and Music
- UK Research and Innovation
- European Research Council
Zitieren
APA
Zhang, H., Chowdhury, S., Cancino-Chacón, C. E., Liang, J., Dixon, S., & Widmer, G. (2024). DExter: Learning and Controlling Performance Expression with Diffusion Models. Applied Sciences, 14(15), 6543. https://doi.org/10.3390/app14156543
Vancouver
Zhang H, Chowdhury S, Cancino-Chacón CE, Liang J, Dixon S, Widmer G. DExter: Learning and Controlling Performance Expression with Diffusion Models. Applied Sciences. 2024;14(15):6543. doi:10.3390/app14156543
BibTeX
@article{Zhang2024,
author = {Zhang, Huan and Chowdhury, Shreyan and Cancino-Chacón, Carlos Eduardo and Liang, Jinhua and Dixon, Simon and Widmer, Gerhard},
title = {DExter: Learning and Controlling Performance Expression with Diffusion Models},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {15},
pages = {6543},
doi = {10.3390/app14156543}
}
EUVIMED