EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Evaluating the Impact of Behavioural Features on Hindi Speech Emotion Recognition: A Multimodal Deep Learning Approach

Sujata Kotian, Santosh Singh

International Journal of Applied Artificial Intelligence and Robotics · 2026

DOI: 10.67745/ijaic.v2i1.9 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Applied Artificial Intelligence and Robotics, 2026. Verfasst von Sujata Kotian und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Context

Speech Emotion Recognition (SER) is an important part of affective computing, but that it cannot work effectively in low resource languages like Hindi. The available SER systems have focused on low-level speech features (acoustic and prosodic) and little has been done to investigate the high-level behavioural speech features (e.g., pauses and rhythm) even though they are significant in human emotional communication.

Objective

This study aimed to explore the hypothesis, whether explicit behavioral speech features can enhance Hindi SER performance, as well as study their joint role in complementing acoustic and prosodic features within a multimodal deep learning system.

Method

A curated Hindi emotional speech corpus of 2,370 utterances of 25 speakers composed of seven emotion classes was studied through a controlled experimental study. The acoustic, prosodic and behavioral features were obtained and represented with a dual-branch multimodal deep learning framework that included CNN/transformer and BiLSTM-attention modules.

Results

The entire multimodal model had an accuracy of 83.9% and a macro-F1 of 0.81, which was significantly higher than the acoustics-only and acoustics-prosodic baselines. The behavioral features provided significant progress to low-arousal emotions, including sadness and neutral, and medium to large effect sizes.

Conclusions

The results show that Hindi SER accuracy and strength is significantly increased by behavioral speech cues. To practitioners, the findings provide justification to apply behavior-aware SER in practice, whereas to researchers, they show the necessity to explicitly model the behavioral characteristics in low-resource and culturally diverse languages.

Zitieren

APA

Kotian, S., & Singh, S. (2026). Evaluating the Impact of Behavioural Features on Hindi Speech Emotion Recognition: A Multimodal Deep Learning Approach. International Journal of Applied Artificial Intelligence and Robotics. https://doi.org/10.67745/ijaic.v2i1.9

Vancouver

Kotian S, Singh S. Evaluating the Impact of Behavioural Features on Hindi Speech Emotion Recognition: A Multimodal Deep Learning Approach. International Journal of Applied Artificial Intelligence and Robotics. 2026. doi:10.67745/ijaic.v2i1.9

BibTeX
@article{Kotian2026,
  author = {Kotian, Sujata and Singh, Santosh},
  title = {Evaluating the Impact of Behavioural Features on Hindi Speech Emotion Recognition: A Multimodal Deep Learning Approach},
  journal = {International Journal of Applied Artificial Intelligence and Robotics},
  year = {2026},
  doi = {10.67745/ijaic.v2i1.9}
}
Kostenlos starten →