EUVIMEDEUVIMED entdecken
Crossref

MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition

Zheng Lian, Licai Sun, Yong Ren, Hao Gu, Haiyang Sun, Lan Chen, Bin Liu, Jianhua Tao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-05 · 48 (5) · 5793-5810

DOI: 10.1109/tpami.2026.3653457 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-05. Verfasst von Zheng Lian und 7 weiteren Autor:innen. Hauptthemen laut MEDLINE: Pattern Recognition, Automated, Emotions, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 100 Literaturangaben. Gefördert von: Excellent Youth Program of the State Key Laboratory of Multimodal Artificial Intelligence Systems, National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal emotion recognition plays a vital role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have developed a range of algorithms and made remarkable progress. While each approach demonstrates certain advantages, inconsistent choices in feature extraction methods, evaluation protocols, and experimental settings have hindered fair comparisons among them. These inconsistencies significantly impede the advancement of the field. To address this issue, we introduce MERBench, a unified evaluation benchmark for multimodal emotion recognition. Our goal is to assess the contributions of several key techniques commonly used in prior studies, such as feature selection, multimodal fusion, robustness analysis, fine-tuning, and pre-training. We believe this work offers clear and comprehensive guidance for future research. Based on the evaluation results of MERBench, we further point out some promising research directions. In addition, we present a new emotion dataset, MER2023, specifically designed for the Chinese language environment. This dataset serves as a benchmark for research in multi-label learning, noise robustness, and semi-supervised learning.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Pattern Recognition, AutomatedEmotionsImage Processing, Computer-AssistedHumansAlgorithmsBenchmarkingDatabases, Factual

Förderung

  • Excellent Youth Program of the State Key Laboratory of Multimodal Artificial Intelligence Systems
  • National Natural Science Foundation of China

Zitieren

APA

Lian, Z., Sun, L., Ren, Y., Gu, H., Sun, H., Chen, L., Liu, B., & Tao, J. (2026). MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(5), 5793-5810. https://doi.org/10.1109/tpami.2026.3653457

Vancouver

Lian Z, Sun L, Ren Y, Gu H, Sun H, Chen L, et al. MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(5):5793-5810. doi:10.1109/tpami.2026.3653457

BibTeX
@article{Lian2026,
  author = {Lian, Zheng and Sun, Licai and Ren, Yong and Gu, Hao and Sun, Haiyang and Chen, Lan and Liu, Bin and Tao, Jianhua},
  title = {MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {5},
  pages = {5793--5810},
  doi = {10.1109/tpami.2026.3653457}
}
Kostenlos starten →