ARMNet: A Network for Image Dimensional Emotion Prediction Based on Affective Region Extraction and Multi-Channel Fusion
Sensors · 2024
DOI: 10.3390/s24217099 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Jingjing Zhang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Communication University of China, Beij…. Hauptthemen laut MEDLINE: Emotions, Neural Networks, Computer. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
Compared with discrete emotion space, image emotion analysis based on dimensional emotion space can more accurately represent fine-grained emotion. Meanwhile, this high-precision representation of emotion requires dimensional emotion prediction methods to sense and capture emotional information in images as accurately and richly as possible. However, the existing methods mainly focus on emotion recognition by extracting the emotional regions where salient objects are located while ignoring the joint influence of objects and background on emotion. Furthermore, in the existing literature, when fusing multi-level features, no consideration has been given to the varying contributions of features from different levels to emotional analysis, which makes it difficult to distinguish valuable and useless features and cannot improve the utilization of effective features. This paper proposes an image emotion prediction network named ARMNet. In ARMNet, a unified affective region extraction method that integrates eye fixation detection and attention detection is proposed to enhance the combined influence of objects and backgrounds. Additionally, the multi-level features are fused with the consideration of their different contributions through an improved channel attention mechanism. In comparison to the existing methods, experiments conducted on the CGnA10766 dataset demonstrate that the performance of valence and arousal, as measured by Mean Squared Error (MSE), Mean Absolute Error (MAE), and Coefficient of Determination (R²), has improved by 4.74%, 3.53%, 3.62%, 1.93%, 6.29%, and 7.23%, respectively. Furthermore, the interpretability of the network is enhanced through the visualization of attention weights corresponding to emotional regions within the images.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Communication University of China, Beijing 100024, China.
- School of Computer and Cyber Sciences, Communication University of China, Beijing 100024, China.
- Beijing Key Laboratory of Modern Entertainment Technology, Communication University of China, Beijing 100024, China.
- Center for Ethnic and Folk Literature and Art Development, Ministry of Culture and Tourism, Beijing 100007, China.
Förderung
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Zhang, J., Sun, J., Wang, C., Tao, Z., & Zhang, F. (2024). ARMNet: A Network for Image Dimensional Emotion Prediction Based on Affective Region Extraction and Multi-Channel Fusion. Sensors. https://doi.org/10.3390/s24217099
Vancouver
Zhang J, Sun J, Wang C, Tao Z, Zhang F. ARMNet: A Network for Image Dimensional Emotion Prediction Based on Affective Region Extraction and Multi-Channel Fusion. Sensors. 2024. doi:10.3390/s24217099
BibTeX
@article{Zhang2024,
author = {Zhang, Jingjing and Sun, Jiaying and Wang, Chunxiao and Tao, Zui and Zhang, Fuxiao},
title = {ARMNet: A Network for Image Dimensional Emotion Prediction Based on Affective Region Extraction and Multi-Channel Fusion},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24217099}
}
EUVIMED