EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU

Xiangsen Zhang, Zhongqiang Wu, Ke Liu, Zengshun Zhao, Jinhao Wang, Chengqin Wu

Sensors · 28.01.2023 · 23 (3) · 1481

DOI: 10.3390/s23031481 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 28.01.2023. Verfasst von Xiangsen Zhang und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590, China. Die Arbeit wurde bisher 61-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Shandong Province, China Postdoctoral Science Special Foundation Funded Project.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the task of text sentiment analysis, the main problem that we face is that the traditional word vectors represent lack of polysemy, the Recurrent Neural Network cannot be trained in parallel, and the classification accuracy is not high. We propose a sentiment classification model based on the proposed Sliced Bidirectional Gated Recurrent Unit (Sliced Bi-GRU), Multi-head Self-Attention mechanism, and Bidirectional Encoder Representations from Transformers embedding. First, the word vector representation obtained by the BERT pre-trained language model is used as the embedding layer of the neural network. Then the input sequence is sliced into subsequences of equal length. And the Bi-sequence Gated Recurrent Unit is applied to extract the subsequent feature information. The relationship between words is learned sequentially via the Multi-head Self-attention mechanism. Finally, the emotional tendency of the text is output by the Softmax function. Experiments show that the classification accuracy of this model on the Yelp 2015 dataset and the Amazon dataset is 74.37% and 62.57%, respectively. And the training speed of the model is better than most existing models, which verifies the effectiveness of the model.

Schlagwörter

Stichwörter der Autor:innen

BERT word vectorbidirectional slice-gated recurrent unitmulti-head self-attention mechanismsentiment classification

Autor:innen und Institutionen

  • College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590, China.

Förderung

  • Natural Science Foundation of Shandong Province
  • China Postdoctoral Science Special Foundation Funded Project

Zitieren

APA

Zhang, X., Wu, Z., Liu, K., Zhao, Z., Wang, J., & Wu, C. (2023). Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU. Sensors, 23(3), 1481. https://doi.org/10.3390/s23031481

Vancouver

Zhang X, Wu Z, Liu K, Zhao Z, Wang J, Wu C. Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU. Sensors. 2023;23(3):1481. doi:10.3390/s23031481

BibTeX
@article{Zhang2023,
  author = {Zhang, Xiangsen and Wu, Zhongqiang and Liu, Ke and Zhao, Zengshun and Wang, Jinhao and Wu, Chengqin},
  title = {Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {3},
  pages = {1481},
  doi = {10.3390/s23031481}
}
Kostenlos starten →