EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Cloud-Based Cross-Modal Transformer for Emotion Recognition and Adaptive Human–Computer Interaction

Ziwen Zhong, Zhitao Shu, Yue Zhao

Fundamental Scientific Reports in Multidisciplinary Areas · 10.06.2026 · 2 (02) · 363-373

DOI: 10.66238/fsrma93 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Fundamental Scientific Reports in Multidisciplinary Areas, online erschienen am 10.06.2026. Verfasst von Ziwen Zhong und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Emotion recognition is a fundamental component of next-generation human-computer interaction (HCI), enabling machines to perceive, understand, and respond to users’affective states. However, existing systems often rely on single-modality analysis-such as facial expressions, speech tone, or textual sentiment-resulting in limited robustness and poor generalization in real-world environments. To address these challenges, this study proposes a Cloud-Based Cross-Modal Transformer (CMT) framework for multimodal emotion recognition and adaptive human-computer interaction. The proposed model integrates visual, auditory, and textual signals using pretrained encoders-Vision Transformer (ViT), Wav2Vec2, and BERT-and employs a cross-modal attention mechanism to capture complex interdependencies among heterogeneous features. By leveraging cloud computing infrastructure with distributed training (Kubernetes and TensorFlow Serving), the system enables scalable, low-latency emotion recognition for large-scale user interactions. Experiments conducted on benchmark datasets including IEMOCAP, MELD, and AffectNet demonstrate that the proposed Cloud-Based Cross-Modal Transformer (CMT) achieves state-of-the-art performance, improving the F1-score by 3.0% and reducing cross-entropy loss by 12.9% compared to the best fusion-based baselines. Additionally, cloud deployment evaluations show an average response latency of 128 ms, representing a 35% reduction compared with conventional transformer-based fusion systems. These results confirm that the proposed CMT framework enables efficient, real-time emotion recognition and adaptive feedback in interactive applications such as intelligent customer service, virtual tutoring, and affective computing interfaces.These results highlight the effectiveness and scalability of the proposed CMT framework, marking an important step toward cloud-native affective computing and emotionally intelligent interactive systems.

Zitieren

APA

Zhong, Z., Shu, Z., & Zhao, Y. (2026). A Cloud-Based Cross-Modal Transformer for Emotion Recognition and Adaptive Human–Computer Interaction. Fundamental Scientific Reports in Multidisciplinary Areas, 2(02), 363-373. https://doi.org/10.66238/fsrma93

Vancouver

Zhong Z, Shu Z, Zhao Y. A Cloud-Based Cross-Modal Transformer for Emotion Recognition and Adaptive Human–Computer Interaction. Fundamental Scientific Reports in Multidisciplinary Areas. 2026;2(02):363-373. doi:10.66238/fsrma93

BibTeX
@article{Zhong2026,
  author = {Zhong, Ziwen and Shu, Zhitao and Zhao, Yue},
  title = {A Cloud-Based Cross-Modal Transformer for Emotion Recognition and Adaptive Human–Computer Interaction},
  journal = {Fundamental Scientific Reports in Multidisciplinary Areas},
  year = {2026},
  volume = {2},
  number = {02},
  pages = {363--373},
  doi = {10.66238/fsrma93}
}
Kostenlos starten →