EUVIMEDEUVIMED entdecken
Crossref

AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing

Vanitha A, Mohammed Roshan Akther K M

IJARCCE · 08.04.2026 · 15 (4)

DOI: 10.17148/ijarcce.2026.15424 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IJARCCE, online erschienen am 08.04.2026. Verfasst von Vanitha A und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Contemporary emotion recognition systems predominantly depend on Convolutional Neural Network (CNN) classifiers pre-trained on constrained label sets, rendering them brittle in unconstrained real-world conditions.This paper presents an AI-powered multimodal emotion recognition framework that exploits the zero-shot reasoning capability of the Google Gemini 1.5 Flash Large Multimodal Model (LMM) to perform high-fidelity facial micro-expression analysis directly in a web browser.The system captures live video frames using the HTML5 MediaDevices API, performs clientside JPEG compression via the Canvas API, and transmits Base64-encoded image payloads to the Gemini inference endpoint, along with a carefully engineered multimodal prompt.Dynamic JSON schema enforcement ensures structured, type-safe responses that contain the detected emotion, a contextual explanation, bounding-box coordinates (normalized to a 0-1000 scale), and an affective quote.The front-end is implemented using React 19, Vite 6, TypeScript, Tailwind CSS, and Framer Motion, forming a production-grade, Decoupled, Multimodal Architecture.Experimental evaluation demonstrates a mean round-trip inference latency of 1.3 seconds on broadband connectivity, as well as the ability to detect fine-grained affective states well beyond the canonical "Big Six" categories, including complex states such as "Suppressed Anger" and "Cautiously Optimistic."The architecture adheres to a strict zero-storage privacy model compliant with GDPR and CCPA.Results confirm that cloud-based LMM prompting supersedes CNN edge models for nuanced emotion understanding while remaining accessible via commodity hardware.

Zitieren

APA

A, V., & M, M. R. A. K. (2026). AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing. IJARCCE, 15(4). https://doi.org/10.17148/ijarcce.2026.15424

Vancouver

A V, M MRAK. AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing. IJARCCE. 2026;15(4). doi:10.17148/ijarcce.2026.15424

BibTeX
@article{A2026,
  author = {A, Vanitha and M, Mohammed Roshan Akther K},
  title = {AI-Powered Multimodal Emotion Recognition: A Zero-Shot Framework Using Large Multimodal Models for Real-Time Affective Computing},
  journal = {IJARCCE},
  year = {2026},
  volume = {15},
  number = {4},
  doi = {10.17148/ijarcce.2026.15424}
}
Kostenlos starten →