Frame and Utterance Emotional Alignment for Speech Emotion Recognition
Future Internet · 05.11.2025 · 17 (11) · 509
DOI: 10.3390/fi17110509 ↗Kurz gefasst
Fachartikel in der Zeitschrift Future Internet, online erschienen am 05.11.2025. Verfasst von Seounghoon Byun und 1 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Speech Emotion Recognition (SER) is important for applications such as Human–Computer Interaction (HCI) and emotion-aware services. Traditional SER models rely on utterance-level labels, aggregating frame-level representations through pooling operations. However, emotional states can vary across frames within an utterance, making it difficult for models to learn consistent and robust representations. To address this issue, we propose two auxiliary loss functions, Emotional Attention Loss (EAL) and Frame-to-Utterance Alignment Loss (FUAL). The proposed approach uses a Classification token (CLS) self-attention pooling mechanism, where the CLS summarizes the entire utterance sequence. EAL encourages frames of the same emotion to align closely with the CLS while separating frames of different classes, and FUAL enforces consistency between frame-level and utterance-level predictions to stabilize training. Model training proceeds in two stages: Stage 1 fine-tunes the wav2vec 2.0 backbone with Cross-Entropy (CE) loss to obtain stable frame embeddings, and stage 2 jointly optimizes CE, EAL and FUAL within the CLS-based pooling framework. Experiments on the IEMOCAP four-class dataset demonstrate that our method consistently outperforms baseline models, showing that the proposed losses effectively address representation inconsistencies and improve SER performance. This work advances Artificial Intelligence by improving the ability of models to understand human emotions through speech.
Zitieren
APA
Byun, S., & Lee, S. (2025). Frame and Utterance Emotional Alignment for Speech Emotion Recognition. Future Internet, 17(11), 509. https://doi.org/10.3390/fi17110509
Vancouver
Byun S, Lee S. Frame and Utterance Emotional Alignment for Speech Emotion Recognition. Future Internet. 2025;17(11):509. doi:10.3390/fi17110509
BibTeX
@article{Byun2025,
author = {Byun, Seounghoon and Lee, Seok-Pil},
title = {Frame and Utterance Emotional Alignment for Speech Emotion Recognition},
journal = {Future Internet},
year = {2025},
volume = {17},
number = {11},
pages = {509},
doi = {10.3390/fi17110509}
}
EUVIMED