EUVIMEDEUVIMED entdecken
Crossref

A Unified Uncertainty-Aware and Imbalance-Resilient Framework for Robust Multimodal Emotion Recognition

Qasim Mahdi Haref

International Journal of AI and Advanced Computing · 2026

DOI: 10.36099/ijaac.v1i3.002 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of AI and Advanced Computing, 2026. Verfasst von Qasim Mahdi Haref. Die Arbeit nennt 27 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The multimodal emotion recognition has become a key field of research because of its importance in human–computer interaction, affective computing and intelligent decision support systems. But these techniques are generally hampered by modality imbalance, class distribution imbalance and propagation of uncertainty across heterogeneous feature space. To solve these problems, the proposed study provides a unified and modular multimodal learning framework based on four complementary enhancement mechanisms to boost the robustness, generalization, and interpretability. The proposed framework consists of (i) an uncertainty-aware adaptive gating mechanism to control the contributions from the different modality, (ii) a multi-sample dropout strategy for better regularization and to prevent overfitting, (iii) a contrastive representation alignment module to ensure the consistency of the different modalities, and (iv) a logit-adjusted learning scheme to deal with the issue of class imbalance. All are organized as separate components, which allows for flexible activation and for systematic ablation analysis. The proposed framework consistently outperforms baseline emotion models and all individual variants of the proposed enhancement on the weighted F1-score, macro F1-score, and overall classification accuracy on extensive experiments conducted on a large-scale multimodal emotion dataset. The ablation results also validate the contribution of uncertainty modeling and imbalance-aware learning, as they yield the best performance. Interestingly, the full configuration works well on CPU-based environments and provides the best generalization performance. The work presented in this thesis makes a significant contribution to the scalable, interpretable, and practically deployable multimodal emotion recognition framework, and provides insights into future research for developing adaptive multimodal learning and powerful affective computing systems.

Zitieren

APA

Haref, Q. M. (2026). A Unified Uncertainty-Aware and Imbalance-Resilient Framework for Robust Multimodal Emotion Recognition. International Journal of AI and Advanced Computing. https://doi.org/10.36099/ijaac.v1i3.002

Vancouver

Haref QM. A Unified Uncertainty-Aware and Imbalance-Resilient Framework for Robust Multimodal Emotion Recognition. International Journal of AI and Advanced Computing. 2026. doi:10.36099/ijaac.v1i3.002

BibTeX
@article{Haref2026,
  author = {Haref, Qasim Mahdi},
  title = {A Unified Uncertainty-Aware and Imbalance-Resilient Framework for Robust Multimodal Emotion Recognition},
  journal = {International Journal of AI and Advanced Computing},
  year = {2026},
  doi = {10.36099/ijaac.v1i3.002}
}
Kostenlos starten →