eXCube2: Explainable Brain-Inspired Spiking Neural Network Framework for Emotion Recognition from Audio, Visual and Multimodal Audio–Visual Data
Biomimetics · 14.03.2026 · 11 (3) · 208
DOI: 10.3390/biomimetics11030208 ↗Kurz gefasst
Fachartikel in der Zeitschrift Biomimetics, online erschienen am 14.03.2026. Verfasst von N K Kasabov und 5 weiteren Autor:innen. Beteiligte Einrichtung: Knowledge Engineering and Discovery Research Institute (KEDRI), School of Engineering, Computer and Mathematical Sciences, Auckland University of Tec…. Die Arbeit nennt 88 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This paper introduces a biomimetic framework and novel brain-inspired AI (BIAI) models based on spiking neural networks (SNNs) for emotional state recognition from audio (speech), visual (face), and integrated multimodal audio–visual data. The developed framework, named eXCube2, uses a three-dimensional SNN architecture NeuCube that is spatially structured according to a human brain template. The BIAI models developed in eXCube2 are trainable on spatio- and spectro-temporal data using brain-inspired learning rules. Such models are explainable in terms of revealing patterns in data and are adaptable to new data. The eXCube2 models are implemented as software systems and tested on speech and video data of subjects expressing emotional states. The use of a brain template for the SNN structure enables brain-inspired tonotopic and stereo mapping of audio inputs, topographic mapping of visual data, and the combined use of both modalities. This novel approach brings AI-based emotional state recognition closer to human perception, provides a better explainability and adaptability than existing AI systems. It also results in a higher or competitive accuracy, even though this was not the main goal here. This is demonstrated through experiments on benchmark datasets, achieving classification accuracy above 80% on single-modality data and 88.9% when multimodal audio–visual data are used, and a “don’t know” output is introduced. The paper further discusses possible applications of the proposed eXCube2 framework to other audio, visual, and audio–visual data for solving challenging problems, such as recognizing emotional states of people from different origins; brain state diagnosis (e.g., Parkinson’s disease, Alzheimer’s disease, ADHD, dementia); measuring response to treatment over time; evaluating satisfaction responses from online clients; cognitive robotics; human–robot interaction; chatbots; and interactive computer games. The SNN-based implementation of BIAI also enables the use of neuromorphic chips and platforms, leading to reduced power consumption, smaller device size, higher performance accuracy, and improved adaptability and explainability. This research shows a step toward building brain-inspired AI systems.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Knowledge Engineering and Discovery Research Institute (KEDRI), School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology (AUT), WZ Building, St. Paul Street, Auckland 1010, New Zealan…
- Institute for Information and Communication Technologies (IICT), Bulgarian Academy of Sciences, Acad. Georgi Bonchev Str., Bl. 2,25A, 1113 Sofia, Bulgaria.
- KEC Ltd., 47C Nihill Cr., Auckland 1071, New Zealand.
- ManaBridge Ltd., Queenstown 9371, New Zealand.
- Department of Industrial Applications of Computers, Technical University of Sofia, ul. "Professor Georgi Bradistilov" 11, 1756 Sofia, Bulgaria.
- Department of Marketing and Communication, School of Business Administration, Athens University of Economics and Business, 28is Oktovriou 76, 104 34 Athina, Greece.
ORCID-Profile: N K Kasabov, T Lappas
Zitieren
APA
Kasabov, N. K., Yang, A., Wang, Z., Abouhassan, I., Kassabova, A., & Lappas, T. (2026). eXCube2: Explainable Brain-Inspired Spiking Neural Network Framework for Emotion Recognition from Audio, Visual and Multimodal Audio–Visual Data. Biomimetics, 11(3), 208. https://doi.org/10.3390/biomimetics11030208
Vancouver
Kasabov NK, Yang A, Wang Z, Abouhassan I, Kassabova A, Lappas T. eXCube2: Explainable Brain-Inspired Spiking Neural Network Framework for Emotion Recognition from Audio, Visual and Multimodal Audio–Visual Data. Biomimetics. 2026;11(3):208. doi:10.3390/biomimetics11030208
BibTeX
@article{Kasabov2026,
author = {Kasabov, N K and Yang, A and Wang, Z and Abouhassan, I and Kassabova, A and Lappas, T},
title = {eXCube2: Explainable Brain-Inspired Spiking Neural Network Framework for Emotion Recognition from Audio, Visual and Multimodal Audio–Visual Data},
journal = {Biomimetics},
year = {2026},
volume = {11},
number = {3},
pages = {208},
doi = {10.3390/biomimetics11030208}
}
EUVIMED