EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Multimodal Emotion Detection for Education and Work Environment by using Improved Artificial Intelligence Machine Vision System

Wan Mohd Bukhari Daud, Adnan Kiral, Mohamed Osman Tokhi, Lee Chung Yee, Muhammad Muzhafar Mohammad Zawawi

Journal of Automation, Mobile Robotics and Intelligent Systems · 10.06.2026 · 53-62

DOI: 10.14313/jamris-2026-019 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Automation, Mobile Robotics and Intelligent Systems, online erschienen am 10.06.2026. Verfasst von Wan Mohd Bukhari Daud und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The use of Artificial Intelligence (AI) has significantly advanced emotion recognition within Human-Computer Interaction (HCI). This paper aims to develop a multimodal emotion detection system for educational and work environments using an enhanced AI machine vision system. The primary focus is on training and testing a multimodal AI model in Python, utilizing Convolutional Neural Networks (CNN). The results from the trained facial emotion AI model demonstrated substantial improvements. Training accuracy increased from 30.49% to 72.21%, while validation accuracy improved from 37.6% to 60.58%. Simultaneously, training loss decreased from 180.69% to 73.65%, and validation loss reduced from 172.97% to 107.53%. This CNN-based model can detect seven emotions: happy, sad, neutral, angry, fear, disgust, and surprise, using OpenCV. Furthermore, the ECG emotion AI model, also trained with CNN, successfully recognized patterns for the same seven emotions. When these two models are combined into a multimodal AI system, they can detect facial and ECG-based emotions simultaneously. This comprehensive approach allows for the detection of both visible and hidden emotions, such as stress or anxiety, which may not be easily discernible through facial expressions alone. The integration of these models into a multimodal AI system provides a more accurate and holistic understanding of human emotions, enhancing applications in educational and work settings. The improved detection capabilities can lead to better user experiences and more effective responses to emotional states, ultimately contributing to advancements in HCI.

Zitieren

APA

Daud, W. M. B., Kiral, A., Tokhi, M. O., Yee, L. C., & Zawawi, M. M. M. (2026). Multimodal Emotion Detection for Education and Work Environment by using Improved Artificial Intelligence Machine Vision System. Journal of Automation, Mobile Robotics and Intelligent Systems, 53-62. https://doi.org/10.14313/jamris-2026-019

Vancouver

Daud WMB, Kiral A, Tokhi MO, Yee LC, Zawawi MMM. Multimodal Emotion Detection for Education and Work Environment by using Improved Artificial Intelligence Machine Vision System. Journal of Automation, Mobile Robotics and Intelligent Systems. 2026:53-62. doi:10.14313/jamris-2026-019

BibTeX
@article{Daud2026,
  author = {Daud, Wan Mohd Bukhari and Kiral, Adnan and Tokhi, Mohamed Osman and Yee, Lee Chung and Zawawi, Muhammad Muzhafar Mohammad},
  title = {Multimodal Emotion Detection for Education and Work Environment by using Improved Artificial Intelligence Machine Vision System},
  journal = {Journal of Automation, Mobile Robotics and Intelligent Systems},
  year = {2026},
  pages = {53--62},
  doi = {10.14313/jamris-2026-019}
}
Kostenlos starten →