EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Interactive behaviors recognition and teaching decision-making optimization in the English classroom under multimodal data integration

Xian Ling Niu

Data and Metadata · 16.04.2026 · 5 · 809

DOI: 10.56294/dm2026809 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Data and Metadata, online erschienen am 16.04.2026. Verfasst von Xian Ling Niu. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

The increasing adoption of multimodal data analytics in educational settings presents significant opportunities to enhance classroom interaction and improve teaching effectiveness. However, traditional methods that rely on single-modality inputs or simple feature concatenation often fail to accurately recognize student behavior and provide actionable teaching recommendations.Objective: aims to develop a comprehensive multimodal framework that accurately identifies interactive student behaviors and enhances teaching decision-making through the integration of audio, visual, and textual data.

Method

The proposed framework utilizes the Student Psychology-Optimized Bilinear Multimodal Network (SPB-MN) to dynamically tune model parameters and adapt to classroom environments. Bilinear feature modeling is employed to capture cross-modal correlations, while a deep neural network backbone with soft attention-based fusion integrates multimodal features. Preprocessing techniques include Gaussian filtering for visual noise reduction, audio normalization, tokenization, and text normalization. Feature extraction is performed using Mel-Frequency Cepstral Coefficients (MFCC) for audio, Bag-of-Words (BoW) for text, and convolutional neural network (CNN) embeddings for visual data. Each modality encoder extracts discriminative features, which are fused through attention-driven bilinear layers to form a unified representation. The system is implemented using Python with TensorFlow and PyTorch and evaluated on classroom datasets consisting of video, audio, transcripts, and interaction logs.Results: the findings show that the proposed framework reaches a high behavior recognition accuracy of 98,85 %, outperforming conventional multimodal approaches and significantly improving teaching decision support.Conclusion: the SPB-MN framework provides a novel and effective solution for multimodal classroom analytics, offering interpretable insights for monitoring student interactions and enabling adaptive teaching optimization.

Zitieren

APA

Niu, X. L. (2026). Interactive behaviors recognition and teaching decision-making optimization in the English classroom under multimodal data integration. Data and Metadata, 5, 809. https://doi.org/10.56294/dm2026809

Vancouver

Niu XL. Interactive behaviors recognition and teaching decision-making optimization in the English classroom under multimodal data integration. Data and Metadata. 2026;5:809. doi:10.56294/dm2026809

BibTeX
@article{Niu2026,
  author = {Niu, Xian Ling},
  title = {Interactive behaviors recognition and teaching decision-making optimization in the English classroom under multimodal data integration},
  journal = {Data and Metadata},
  year = {2026},
  volume = {5},
  pages = {809},
  doi = {10.56294/dm2026809}
}
Kostenlos starten →