MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Classification Tasks
Journal of Creative Music Systems · 2024
DOI: 10.5920/jcms.1064 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Creative Music Systems, 2024. Verfasst von Yi-Hui Chou und 4 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This paper presents a benchmark study of MIDI-domain music classification using the mask language modeling approach of the Bidirectional Encoder Representations from Transformers (BERT). Specifically, with five public-domain datasets of single-track polyphonic piano MIDI files, we pre-train a 12-layer Transformer model using the BERT approach and fine-tune it for four downstream classification tasks. These include two note-level classification tasks---melody extraction and velocity prediction, and two sequence-level classification tasks---artist classification and emotion classification. Compared to strong recurrent neural network (RNN)-based baselines, the BERT approach does lead to higher classification accuracy, with less than 10 epochs of fine-tuning. Ablation studies further show that the pre-training remains effective even if the MIDI data of the downstream tasks are not seen during the pre-training stage, and that freezing the self-attention layers at the fine-tuning stage only slightly degrades the performance. For reproducibility, we share the weights of our pre-trained and fine-tuned models.
Zitieren
APA
Chou, Y., Chen, I., Ching, J., Chang, C., & Yang, Y. (2024). MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Classification Tasks. Journal of Creative Music Systems. https://doi.org/10.5920/jcms.1064
Vancouver
Chou Y, Chen I, Ching J, Chang C, Yang Y. MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Classification Tasks. Journal of Creative Music Systems. 2024. doi:10.5920/jcms.1064
BibTeX
@article{Chou2024,
author = {Chou, Yi-Hui and Chen, I-Chun and Ching, Joann and Chang, Chin-Jui and Yang, Yi-Hsuan},
title = {MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Classification Tasks},
journal = {Journal of Creative Music Systems},
year = {2024},
doi = {10.5920/jcms.1064}
}
EUVIMED