A Course-Review Sentiment Analysis Method Integrating Cross-Head Attention and Lightweight Knowledge Distillation
International Journal of All Research Education and Scientific Methods · 2026 · 14 (11) · 465-471
DOI: 10.56025/ijaresm.14112658 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of All Research Education and Scientific Methods, 2026. Verfasst von Chenjie Du und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Sentiment classification of Chinese course reviews requires a model that captures contextual cues at a practical inference cost. We combine a four-layer BERT student with cross multi-head attention (CMHA) and output level knowledge distillation from BERT-base. CMHA refines the aggregated attention output through a residual projection, and distillation combines temperature-scaled distribution matching with supervision from the original labels. We evaluate the model on ChnSentiCorp, Waimai_10k and EduTE-Putter, a collection of 78,447 Chinese course reviews. In the ablation experiments, the combined model improved accuracy over BERT-mini by 2.3, 3.4 and 2.1 percentage points, respectively. In the EduTE-Putter model comparison, the student achieved 79.6% accuracy with 23.95 million parameters, compared with 82.4% and 102.27 million parameters for BERT base. The results quantify the trade-off between accuracy and model size in course-review sentiment classification. Compared with BERT-base, this is a 76.6% reduction in model parameters. The student also processed reviews faster than the other models in the reported comparison. The ablations show that the two components yield complementary accuracy gains, although their combination does not always give the highest throughput.
Zitieren
APA
Du, C., Shen, W., Ye, J., Zhou, J., & Tian, S. (2026). A Course-Review Sentiment Analysis Method Integrating Cross-Head Attention and Lightweight Knowledge Distillation. International Journal of All Research Education and Scientific Methods, 14(11), 465-471. https://doi.org/10.56025/ijaresm.14112658
Vancouver
Du C, Shen W, Ye J, Zhou J, Tian S. A Course-Review Sentiment Analysis Method Integrating Cross-Head Attention and Lightweight Knowledge Distillation. International Journal of All Research Education and Scientific Methods. 2026;14(11):465-471. doi:10.56025/ijaresm.14112658
BibTeX
@article{Du2026,
author = {Du, Chenjie and Shen, Wei and Ye, Jiahui and Zhou, Junjie and Tian, Shengzheng},
title = {A Course-Review Sentiment Analysis Method Integrating Cross-Head Attention and Lightweight Knowledge Distillation},
journal = {International Journal of All Research Education and Scientific Methods},
year = {2026},
volume = {14},
number = {11},
pages = {465--471},
doi = {10.56025/ijaresm.14112658}
}
EUVIMED