EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-SA 4.0

Contrastive Learning Models for Sentence Representations

Lingling Xu, Haoran Xie, Zongxi Li, Fu Lee Wang, Weiming Wang, Qing Li

ACM Transactions on Intelligent Systems and Technology · 31.08.2023 · 14 (4) · 1-34

DOI: 10.1145/3593590 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Intelligent Systems and Technology, online erschienen am 15.06.2023. Verfasst von Lingling Xu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 52-mal zitiert (Crossref) und nennt 142 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0). Gefördert von: Research Grants Council of the Hong Kong Special Administrative Region, China, Lam Woo Research Fund of Lingnan University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sentence representation learning is a crucial task in natural language processing, as the quality of learned representations directly influences downstream tasks, such as sentence classification and sentiment analysis. Transformer-based pretrained language models such as bidirectional encoder representations from transformers (BERT) have been extensively applied to various natural language processing tasks, and have exhibited moderately good performance. However, the anisotropy of the learned embedding space prevents BERT sentence embeddings from achieving good results in the semantic textual similarity tasks. It has been shown that contrastive learning can alleviate the anisotropy problem and significantly improve sentence representation performance. Therefore, there has been a surge in the development of models that utilize contrastive learning to fine-tune BERT-like pretrained language models to learn sentence representations. But no systematic review of contrastive learning models for sentence representations has been conducted. To fill this gap, this article summarizes and categorizes the contrastive learning based sentence representation models, common evaluation tasks for assessing the quality of learned representations, and future research directions. Furthermore, we select several representative models for exhaustive experiments to illustrate the quantitative improvement of various strategies on sentence representations.

Förderung

  • Research Grants Council of the Hong Kong Special Administrative Region, China
  • Lam Woo Research Fund of Lingnan University

Zitieren

APA

Xu, L., Xie, H., Li, Z., Wang, F. L., Wang, W., & Li, Q. (2023). Contrastive Learning Models for Sentence Representations. ACM Transactions on Intelligent Systems and Technology, 14(4), 1-34. https://doi.org/10.1145/3593590

Vancouver

Xu L, Xie H, Li Z, Wang FL, Wang W, Li Q. Contrastive Learning Models for Sentence Representations. ACM Transactions on Intelligent Systems and Technology. 2023;14(4):1-34. doi:10.1145/3593590

BibTeX
@article{Xu2023,
  author = {Xu, Lingling and Xie, Haoran and Li, Zongxi and Wang, Fu Lee and Wang, Weiming and Li, Qing},
  title = {Contrastive Learning Models for Sentence Representations},
  journal = {ACM Transactions on Intelligent Systems and Technology},
  year = {2023},
  volume = {14},
  number = {4},
  pages = {1--34},
  doi = {10.1145/3593590}
}
Kostenlos starten →