Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU
Sensors · 28.01.2023 · 23 (3) · 1481
DOI: 10.3390/s23031481 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 28.01.2023. Verfasst von Xiangsen Zhang und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590, China. Die Arbeit wurde bisher 61-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Shandong Province, China Postdoctoral Science Special Foundation Funded Project.
Abstract (Originalsprache)
In the task of text sentiment analysis, the main problem that we face is that the traditional word vectors represent lack of polysemy, the Recurrent Neural Network cannot be trained in parallel, and the classification accuracy is not high. We propose a sentiment classification model based on the proposed Sliced Bidirectional Gated Recurrent Unit (Sliced Bi-GRU), Multi-head Self-Attention mechanism, and Bidirectional Encoder Representations from Transformers embedding. First, the word vector representation obtained by the BERT pre-trained language model is used as the embedding layer of the neural network. Then the input sequence is sliced into subsequences of equal length. And the Bi-sequence Gated Recurrent Unit is applied to extract the subsequent feature information. The relationship between words is learned sequentially via the Multi-head Self-attention mechanism. Finally, the emotional tendency of the text is output by the Softmax function. Experiments show that the classification accuracy of this model on the Yelp 2015 dataset and the Amazon dataset is 74.37% and 62.57%, respectively. And the training speed of the model is better than most existing models, which verifies the effectiveness of the model.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590, China.
Förderung
- Natural Science Foundation of Shandong Province
- China Postdoctoral Science Special Foundation Funded Project
Zitieren
APA
Zhang, X., Wu, Z., Liu, K., Zhao, Z., Wang, J., & Wu, C. (2023). Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU. Sensors, 23(3), 1481. https://doi.org/10.3390/s23031481
Vancouver
Zhang X, Wu Z, Liu K, Zhao Z, Wang J, Wu C. Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU. Sensors. 2023;23(3):1481. doi:10.3390/s23031481
BibTeX
@article{Zhang2023,
author = {Zhang, Xiangsen and Wu, Zhongqiang and Liu, Ke and Zhao, Zengshun and Wang, Jinhao and Wu, Chengqin},
title = {Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU},
journal = {Sensors},
year = {2023},
volume = {23},
number = {3},
pages = {1481},
doi = {10.3390/s23031481}
}
EUVIMED