Audio-Visual Emotion Classification Using Reinforcement Learning-Enhanced Particle Swarm Optimisation
IEEE Transactions on Affective Computing · 2025-10 · 16 (4) · 3434-3451
DOI: 10.1109/taffc.2025.3591356 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-10. Verfasst von Karolis Kondrotas und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 87 Literaturangaben.
Abstract (Originalsprache)
The extraction of fine-grained spatial-temporal characteristics for emotion classification is a challenging task owing to the subtlety and ambiguity of emotional expressions through video and audio channels. In this research, we propose an audio-visual ensemble model, comprising a two-stream 3D Convolutional Neural Network (CNN) architecture with RGB and optical flow as inputs for video emotion classification, as well as a variant of Wav2Vec2 for audio emotion recognition. The Wav2Vec2 variant integrates additional recurrent and attention layers with each transformer block to extract long- and short-term dependencies. A new Particle Swarm Optimisation (PSO) algorithm is proposed to fine-tune hyper-parameters of 3D CNNs and the enhanced Wav2Vec2, and formulate audio-visual ensemble models with the smallest sizes. It integrates a reinforcement learning (RL) algorithm, i.e. Asynchronous Advantage Actor-Critic (A3C), for search parameter and hybrid leader construction, and another RL algorithm, Proximal Policy Optimisation (PPO), for search action selection, as well as hypotrochoid and super formula-based search operations. Evaluated using audio-visual emotion datasets, our evolving ensemble model outperforms those devised by other search methods and existing state-of-the-art deep networks, significantly.
Zitieren
APA
Kondrotas, K., Zhang, L., Lim, C. P., Asadi, H., & Yu, Y. (2025). Audio-Visual Emotion Classification Using Reinforcement Learning-Enhanced Particle Swarm Optimisation. IEEE Transactions on Affective Computing, 16(4), 3434-3451. https://doi.org/10.1109/taffc.2025.3591356
Vancouver
Kondrotas K, Zhang L, Lim CP, Asadi H, Yu Y. Audio-Visual Emotion Classification Using Reinforcement Learning-Enhanced Particle Swarm Optimisation. IEEE Transactions on Affective Computing. 2025;16(4):3434-3451. doi:10.1109/taffc.2025.3591356
BibTeX
@article{Kondrotas2025,
author = {Kondrotas, Karolis and Zhang, Li and Lim, Chee Peng and Asadi, Houshyar and Yu, Yonghong},
title = {Audio-Visual Emotion Classification Using Reinforcement Learning-Enhanced Particle Swarm Optimisation},
journal = {IEEE Transactions on Affective Computing},
year = {2025},
volume = {16},
number = {4},
pages = {3434--3451},
doi = {10.1109/taffc.2025.3591356}
}
EUVIMED