Speech emotion recognition using fine-tuned Wav2vec2.0 and neural controlled differential equations classifier
PLOS ONE · 20.02.2025 · 20 (2) · e0318297
DOI: 10.1371/journal.pone.0318297 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 20.02.2025. Verfasst von Ni Wang und 1 weiteren Autor:innen. Beteiligte Einrichtung: College of Mathematics and Statistics, Chongqing University, Chongqing, China. Hauptthemen laut MEDLINE: Emotions, Speech, Speech Perception. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Chongqing University, Engineering and Physical Sciences Research Council.
Abstract (Originalsprache)
Speech emotion recognition (SER) has always been a popular yet challenging task with broad applications in areas such as social media communication and medical diagnostics. Due to the characteristics of speech emotion recognition dataset, which often have small data volumes and high complexity, effectively integrating and modeling audio data remains a significant challenge in this field. To address this, we propose a model architecture that combines fine-tuned Wave2vec2.0 with Neural Controlled Differential Equations (NCDEs): First, we use a fine-tuned Wav2vec2.0 to extract rich contextual information. Then we model the high-dimensional time series feature set using a Neural Controlled Differential Equation classifier. We set the vector field as an MLP and update the model’s hidden state by solving the controlled differential equation. We conducted speech emotion recognition experiments on the IEMOCAP dataset. The experiments show that our model achieves the weighted accuracy of 73.37% and the unweighted accuracy of 74.18%. Additionally, our model converges very quickly, reaching a good accuracy after just one epoch of training. Furthermore, our model exhibits excellent stability. The standard deviation of weighted accuracy (WA) is 0.45% and the standard deviation of unweighted accuracy (UA) is 0.39%.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- College of Mathematics and Statistics, Chongqing University, Chongqing, China.
ORCID-Profile: Danyu Yang
Förderung
- National Natural Science Foundation of China
- Chongqing University
- Engineering and Physical Sciences Research Council
Zitieren
APA
Wang, N., & Yang, D. (2025). Speech emotion recognition using fine-tuned Wav2vec2.0 and neural controlled differential equations classifier. PLOS ONE, 20(2), e0318297. https://doi.org/10.1371/journal.pone.0318297
Vancouver
Wang N, Yang D. Speech emotion recognition using fine-tuned Wav2vec2.0 and neural controlled differential equations classifier. PLOS ONE. 2025;20(2):e0318297. doi:10.1371/journal.pone.0318297
BibTeX
@article{Wang2025,
author = {Wang, Ni and Yang, Danyu},
title = {Speech emotion recognition using fine-tuned Wav2vec2.0 and neural controlled differential equations classifier},
journal = {PLOS ONE},
year = {2025},
volume = {20},
number = {2},
pages = {e0318297},
doi = {10.1371/journal.pone.0318297}
}
EUVIMED