EUVIMEDEUVIMED entdecken
Crossref

MSMF-MIL: Multi-Scale Mixed Feature-Based Multiple Instance Learning for Speech Emotion Recognition

Dongdong Li, Hongmei Yang, Zihan Song, Zhe Wang

IEEE Transactions on Consumer Electronics · 2025-08 · 71 (3) · 7539-7550

DOI: 10.1109/tce.2025.3596056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Consumer Electronics, 2025-08. Verfasst von Dongdong Li und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Wenzhou Science and Technology Bureau.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Speech emotion in natural utterances is inherently complex and non-uniform, with key emotional cues often confined to brief segments. To address this challenge, we propose a novel multi-scale mixed feature-based framework that leverages Multiple Instance Learning (MIL). In our approach, each speech utterance is transformed into a “bag” containing multiple segments, with each segment treated as an individual instance. Inspired by MIL’s capability to identify key instances within a set, our framework employs CNN-based MIL models at both the frame and utterance-levels, while a ResNet-based model extracts segment-level features. These multi-scale representations are then fused to isolate and emphasize the critical speech segments that express dominant emotions. Experimental evaluations on the Interactive Emotional Dyadic Motion Capture Database (IEMOCAP), the Berlin Database of Emotional Speech (Emo-DB) and the spontaneous URDU-language speech emotion database (URDU) demonstrate our approach’s effectiveness, achieving 67.76% weighted accuracy and 58.39% unweighted accuracy on IEMOCAP, 93.80% weighted accuracy and 91.92% unweighted accuracy on Emo-DB, and 95.75% unweighted accuracy on URDU. These results confirm that our MIL-based, multi-scale feature fusion strategy significantly enhances the robustness and accuracy of speech emotion recognition systems.

Förderung

  • National Natural Science Foundation of China
  • Wenzhou Science and Technology Bureau

Zitieren

APA

Li, D., Yang, H., Song, Z., & Wang, Z. (2025). MSMF-MIL: Multi-Scale Mixed Feature-Based Multiple Instance Learning for Speech Emotion Recognition. IEEE Transactions on Consumer Electronics, 71(3), 7539-7550. https://doi.org/10.1109/tce.2025.3596056

Vancouver

Li D, Yang H, Song Z, Wang Z. MSMF-MIL: Multi-Scale Mixed Feature-Based Multiple Instance Learning for Speech Emotion Recognition. IEEE Transactions on Consumer Electronics. 2025;71(3):7539-7550. doi:10.1109/tce.2025.3596056

BibTeX
@article{Li2025,
  author = {Li, Dongdong and Yang, Hongmei and Song, Zihan and Wang, Zhe},
  title = {MSMF-MIL: Multi-Scale Mixed Feature-Based Multiple Instance Learning for Speech Emotion Recognition},
  journal = {IEEE Transactions on Consumer Electronics},
  year = {2025},
  volume = {71},
  number = {3},
  pages = {7539--7550},
  doi = {10.1109/tce.2025.3596056}
}
Kostenlos starten →