MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-05 · 48 (5) · 5793-5810
DOI: 10.1109/tpami.2026.3653457 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-05. Verfasst von Zheng Lian und 7 weiteren Autor:innen. Hauptthemen laut MEDLINE: Pattern Recognition, Automated, Emotions, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 100 Literaturangaben. Gefördert von: Excellent Youth Program of the State Key Laboratory of Multimodal Artificial Intelligence Systems, National Natural Science Foundation of China.
Abstract (Originalsprache)
Multimodal emotion recognition plays a vital role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have developed a range of algorithms and made remarkable progress. While each approach demonstrates certain advantages, inconsistent choices in feature extraction methods, evaluation protocols, and experimental settings have hindered fair comparisons among them. These inconsistencies significantly impede the advancement of the field. To address this issue, we introduce MERBench, a unified evaluation benchmark for multimodal emotion recognition. Our goal is to assess the contributions of several key techniques commonly used in prior studies, such as feature selection, multimodal fusion, robustness analysis, fine-tuning, and pre-training. We believe this work offers clear and comprehensive guidance for future research. Based on the evaluation results of MERBench, we further point out some promising research directions. In addition, we present a new emotion dataset, MER2023, specifically designed for the Chinese language environment. This dataset serves as a benchmark for research in multi-label learning, noise robustness, and semi-supervised learning.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- Excellent Youth Program of the State Key Laboratory of Multimodal Artificial Intelligence Systems
- National Natural Science Foundation of China
Zitieren
APA
Lian, Z., Sun, L., Ren, Y., Gu, H., Sun, H., Chen, L., Liu, B., & Tao, J. (2026). MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(5), 5793-5810. https://doi.org/10.1109/tpami.2026.3653457
Vancouver
Lian Z, Sun L, Ren Y, Gu H, Sun H, Chen L, et al. MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(5):5793-5810. doi:10.1109/tpami.2026.3653457
BibTeX
@article{Lian2026,
author = {Lian, Zheng and Sun, Licai and Ren, Yong and Gu, Hao and Sun, Haiyang and Chen, Lan and Liu, Bin and Tao, Jianhua},
title = {MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {5},
pages = {5793--5810},
doi = {10.1109/tpami.2026.3653457}
}
EUVIMED