EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MELT: Optimization-Driven Music Emotion Learning with Temporal Token-Level Fusion

Yihe Yin, Zhen Tian, Junming Chen

Mathematics · 15.05.2026 · 14 (10) · 1690

DOI: 10.3390/math14101690 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 15.05.2026. Verfasst von Yihe Yin und 2 weiteren Autor:innen. Die Arbeit nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Fund of Macao University of Science and Technology, Science and Technology Development Fund of Macau Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Music emotion recognition (MER) can be formulated as a multimodal optimization problem that predicts an emotion label from coupled audio and lyric sequences. Existing methods typically perform unimodal learning or coarse global fusion, which overlooks fine-grained temporal-token correspondences between musical dynamics and lyric semantics. We propose MELT (Music Emotion Learning with Temporal token-level fusion), an optimization-driven framework with four modules: a BERT-based lyrics semantic encoder (LSE), a segment temporal encoder (STE) that models audio-segment dependencies via a Transformer, a token-level temporal fusion (TTF) module with gated cross-attention, and an emotion mood head (EMH) for four-class prediction. Training is conducted end-to-end by jointly minimizing a supervised classification term and an auxiliary cross-modal contrastive alignment term, yielding a unified objective that improves both class separability and representation consistency. On the MoodyLyrics benchmark, MELT achieves 87.6% weighted F1 for four-class emotion recognition (angry, happy, relaxed, sad), outperforming unimodal baselines and representative early/late fusion strategies. Ablation results further verify that temporal encoding, gated token-level fusion, and joint optimization each contribute to the final performance.

Förderung

  • Research Fund of Macao University of Science and Technology
  • Science and Technology Development Fund of Macau Project

Zitieren

APA

Yin, Y., Tian, Z., & Chen, J. (2026). MELT: Optimization-Driven Music Emotion Learning with Temporal Token-Level Fusion. Mathematics, 14(10), 1690. https://doi.org/10.3390/math14101690

Vancouver

Yin Y, Tian Z, Chen J. MELT: Optimization-Driven Music Emotion Learning with Temporal Token-Level Fusion. Mathematics. 2026;14(10):1690. doi:10.3390/math14101690

BibTeX
@article{Yin2026,
  author = {Yin, Yihe and Tian, Zhen and Chen, Junming},
  title = {MELT: Optimization-Driven Music Emotion Learning with Temporal Token-Level Fusion},
  journal = {Mathematics},
  year = {2026},
  volume = {14},
  number = {10},
  pages = {1690},
  doi = {10.3390/math14101690}
}
Kostenlos starten →