EUVIMEDEUVIMED entdecken
Crossref

Individual Facial Features Based Video Emotion Recognition

Misbah Ayoub

University of Liverpool · 01.01.2030

DOI: 10.17638/03190647 ↗

Kurz gefasst

Fachartikel in der Zeitschrift University of Liverpool, erschienen am 01.01.2030. Verfasst von Misbah Ayoub.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Facial features play a crucial role in conveying human emotions, as they are arguably key indicators of our emotional state. Changes in eye movement, eyebrow muscles, nostril movement, and mouth opening and closing, as well as overall facial expressions, represent different emotions such as happiness, sadness, anger, and surprise. By analysing facial features, it is possible to identify and interpret various emotional states accurately, making them valuable for applications in psychology, HCI, and automated emotion recognition systems. Recently, there has been rising interest in using machine learning and deep learning models for emotion recognition due to their ability to automatically process complex visual information from raw data. However, these models can require significant computational resources. Their black-box nature reduces feature interpretability, complicating the identification of crucial facial features necessary for effective feature selection. The need for extensive labeled data and resources makes these models less generalizable and more costly for lightweight emotion recognition systems. To address these limitations, a series of models are developed and compared for lightweight and robust individual facial feature-based emotion recognition by applying a systematic approach. Firstly, a fusion-based emotion recognition model combining Convolutional Neural Networks (CNN) and Histogram of Oriented Gradients (HOG) is proposed. This model leverages the complementary strengths of hierarchical and handcrafted features, addressing the limitations of each approach. This model is tested with feed forward neural networks and gradient boost machine learning models. Secondly, for region based facial feature extraction, a straightforward and efficient individual feature extraction algorithm for the upper and lower facial expressions is proposed that improves facial movement detection and recognition. For upper facial expressions, the Lacrimal Aspect Ratio (LAR) feature extraction algorithm for the eyes and Lacrimal Distance Analysis (LDA) for eyebrow movement are introduced. For lower facial expressions, the Nostril Movement Ratio (NMR) algorithm for nostril movement and Mouth Distance Analysis for mouth movement (MDA) are also proposed. Two lightweight machine learning algorithms, ensemble learning and neural networks, are used and compared for classification, including a Convolutional Neural Network and Histogram of Oriented Gradients (CNN-HOG) based approach. This helps identify the strengths and limitations of each feature extraction approach, providing insights into their performance and suitability for different aspects of the task. Additionally, by applying individual facial feature extraction and classification algorithms, we develop a more lightweight and robust model for emotion recognition. Lastly, we comprehensively analyse individual features based on subject, emotions, dataset, illumination, resolution, occlusion handling, and intensity. Moreover, comparison of individual feature extraction algorithms provides a more intuitive and interpretable analysis using ensemble learning and neural network algorithms. The results show that using individual features enhances explainability and efficiency, without relying on dimension reduction models. This proposed approach achieves outcomes comparable to deep learning models, enhancing the recognition model’s capability by integrating crucial details of individual facial features such as eyebrow movement, eye blinking, and nostril movement and mouth movement. This efficiency is maintained with a lightweight configuration on high-resolution images, rather than relying on complex and costly models. The resulting facial emotion recognition model is not only lightweight and robust, relying on geometric features, but also portable and easy to use on any system compared to deep learning-based models. It establishes the basis for a multi-dataset facial emotion framework that is autonomous, optimized, and generalized.

Schlagwörter

video-based emotion recognitionfacial featuresfacial expression analysiseyebrow movementsmouth movementsocclusion handlingensemble learningexplainabilityfeature selectionillumination robustness

Zitieren

APA

Ayoub, M. (2030). Individual Facial Features Based Video Emotion Recognition. University of Liverpool. https://doi.org/10.17638/03190647

Vancouver

Ayoub M. Individual Facial Features Based Video Emotion Recognition. University of Liverpool. 2030. doi:10.17638/03190647

BibTeX
@article{Ayoub2030,
  author = {Ayoub, Misbah},
  title = {Individual Facial Features Based Video Emotion Recognition},
  journal = {University of Liverpool},
  year = {2030},
  doi = {10.17638/03190647}
}
Kostenlos starten →