Short-Text Sentiment Classification Model Based on BERT and Dual-Stream Transformer Gated Attention Mechanism
Electronics · 30.09.2025 · 14 (19) · 3904
DOI: 10.3390/electronics14193904 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 30.09.2025. Verfasst von Song Yang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: 2017 Higher Education Research Project, Dalian University of Foreign Languages.
Abstract (Originalsprache)
With the rapid development of social media, short-text data have become increasingly important in fields such as public opinion monitoring, user feedback analysis, and intelligent recommendation systems. However, existing short-text sentiment analysis models often suffer from limited cross-domain adaptability and poor generalization performance. To address these challenges, this study proposes a novel short-text sentiment classification model based on the Bidirectional Encoder Representations from Transformers (BERTs) and a dual-stream Transformer gated attention mechanism. This model first employs Bidirectional Encoder Representations from Transformers (BERTs) and the Chinese Robustly Optimized BERT Pretraining Approach (Chinese-RoBERTa) to achieve data augmentation and multilevel semantic mining, thereby expanding the training corpus and enhancing minority class coverage. Second, a dual-stream Transformer gated attention mechanism was developed to dynamically adjust feature fusion weights, enhancing adaptability to heterogeneous texts. Finally, the model integrates a Bidirectional Gated Recurrent Unit (BiGRU) with Multi-Head Self-Attention (MHSA) to strengthen sequence information modeling and global context capture, enabling the precise identification of key sentiment dependencies. The model’s superior performance in handling data imbalance and complex textual sentiment logic scenarios is demonstrated by the experimental results, achieving significant improvements in accuracy and F1 score. The F1 score reached 92.4%, representing an average increase of 8.7% over the baseline models. This provides an effective solution for enhancing the performance and expanding the application scenarios of short-text sentiment analysis models.
Förderung
- 2017 Higher Education Research Project
- Dalian University of Foreign Languages
Zitieren
APA
Yang, S., Xing, J., Liu, Z., & Sun, Y. (2025). Short-Text Sentiment Classification Model Based on BERT and Dual-Stream Transformer Gated Attention Mechanism. Electronics, 14(19), 3904. https://doi.org/10.3390/electronics14193904
Vancouver
Yang S, Xing J, Liu Z, Sun Y. Short-Text Sentiment Classification Model Based on BERT and Dual-Stream Transformer Gated Attention Mechanism. Electronics. 2025;14(19):3904. doi:10.3390/electronics14193904
BibTeX
@article{Yang2025,
author = {Yang, Song and Xing, Jiayao and Liu, Zhaoxia and Sun, Yunhao},
title = {Short-Text Sentiment Classification Model Based on BERT and Dual-Stream Transformer Gated Attention Mechanism},
journal = {Electronics},
year = {2025},
volume = {14},
number = {19},
pages = {3904},
doi = {10.3390/electronics14193904}
}
EUVIMED