AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing
IJARCCE · 08.04.2026 · 15 (4)
DOI: 10.17148/ijarcce.2026.15424 ↗Kurz gefasst
Fachartikel in der Zeitschrift IJARCCE, online erschienen am 08.04.2026. Verfasst von Vanitha A und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Contemporary emotion recognition systems predominantly depend on Convolutional Neural Network (CNN) classifiers pre-trained on constrained label sets, rendering them brittle in unconstrained real-world conditions.This paper presents an AI-powered multimodal emotion recognition framework that exploits the zero-shot reasoning capability of the Google Gemini 1.5 Flash Large Multimodal Model (LMM) to perform high-fidelity facial micro-expression analysis directly in a web browser.The system captures live video frames using the HTML5 MediaDevices API, performs clientside JPEG compression via the Canvas API, and transmits Base64-encoded image payloads to the Gemini inference endpoint, along with a carefully engineered multimodal prompt.Dynamic JSON schema enforcement ensures structured, type-safe responses that contain the detected emotion, a contextual explanation, bounding-box coordinates (normalized to a 0-1000 scale), and an affective quote.The front-end is implemented using React 19, Vite 6, TypeScript, Tailwind CSS, and Framer Motion, forming a production-grade, Decoupled, Multimodal Architecture.Experimental evaluation demonstrates a mean round-trip inference latency of 1.3 seconds on broadband connectivity, as well as the ability to detect fine-grained affective states well beyond the canonical "Big Six" categories, including complex states such as "Suppressed Anger" and "Cautiously Optimistic."The architecture adheres to a strict zero-storage privacy model compliant with GDPR and CCPA.Results confirm that cloud-based LMM prompting supersedes CNN edge models for nuanced emotion understanding while remaining accessible via commodity hardware.
Zitieren
APA
A, V., & M, M. R. A. K. (2026). AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing. IJARCCE, 15(4). https://doi.org/10.17148/ijarcce.2026.15424
Vancouver
A V, M MRAK. AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing. IJARCCE. 2026;15(4). doi:10.17148/ijarcce.2026.15424
BibTeX
@article{A2026,
author = {A, Vanitha and M, Mohammed Roshan Akther K},
title = {AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing},
journal = {IJARCCE},
year = {2026},
volume = {15},
number = {4},
doi = {10.17148/ijarcce.2026.15424}
}
EUVIMED