Evaluating significant features in context‐aware multimodal emotion recognition with XAI methods
Expert Systems · 2025-01 · 42 (1)
DOI: 10.1111/exsy.13403 ↗Kurz gefasst
Fachartikel in der Zeitschrift Expert Systems, online erschienen am 08.08.2023. Verfasst von Aaishwarya Khalane und 3 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 72 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Expert systems are being extensively used to make critical decisions involving emotional analysis in affective computing. The evolution of deep learning algorithms has improved the potential for extracting value from multimodal emotional data. However, these black‐box algorithms do not often explain the heuristics behind processing the input features for achieving certain outputs. This study focuses on the risks of using black‐box deep learning models for critical tasks, such as emotion recognition, and describes how human understandable interpretations of the workings of these models are extremely important. This study utilizes one of the largest multimodal datasets available–CMU‐MOSEI. Many researchers have used the pre‐extracted features provided by the CMU Multimodal SDK with black‐box deep learning models making it difficult to interpret the contribution of its individual features. This study describes the implications of significant features from various modalities (audio, video, text) identified using XAI in Multimodal Emotion Recognition. It describes the process of curating reduced feature models by using the Gradient SHAP XAI method. These reduced models with highly contributing features achieve comparable and at times even better results compared to their corresponding all‐feature models as well as the baseline model GraphMFN. This study reveals that carefully selecting significant features for a model can help filter out irrelevant features, and attenuate the noise or bias caused by them, leading to an improved performance efficiency of the expert systems by making them transparent, easily interpretable, and trustworthy.
Zitieren
APA
Khalane, A., Makwana, R., Shaikh, T., & Ullah, A. (2025). Evaluating significant features in context‐aware multimodal emotion recognition with XAI methods. Expert Systems, 42(1). https://doi.org/10.1111/exsy.13403
Vancouver
Khalane A, Makwana R, Shaikh T, Ullah A. Evaluating significant features in context‐aware multimodal emotion recognition with XAI methods. Expert Systems. 2025;42(1). doi:10.1111/exsy.13403
BibTeX
@article{Khalane2025,
author = {Khalane, Aaishwarya and Makwana, Rikesh and Shaikh, Talal and Ullah, Abrar},
title = {Evaluating significant features in context‐aware multimodal emotion recognition with XAI methods},
journal = {Expert Systems},
year = {2025},
volume = {42},
number = {1},
doi = {10.1111/exsy.13403}
}
EUVIMED