EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Integration of multi-level semantics in PTMs with an attention model for question matching

Zheng Ye iD, Linwei Che iD, Jun Ge, Jun Qin, Jing Liu

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0305772 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Zheng Ye und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Computer Science & Information Physics Fusion Intelligent Computing Key Laboratory of the National Ethnic Affairs Commission, South-Centra…. Hauptthemen laut MEDLINE: Semantics, Natural Language Processing. Die Arbeit nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental Research Funds for the Central Universities of South-Central Minzu University, Talent Introduction Program Funds for the Central Universities of South-Central Minzu University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The task of question matching/retrieval focuses on determining whether two questions are semantically equivalent. It has garnered significant attention in the field of natural language processing (NLP) due to its commercial value. While neural network models have made great strides and achieved human-level accuracy, they still face challenges when handling complex scenarios. In this paper, we delve into the utilization of different specializations encoded in different layers of large-scale pre-trained language models (PTMs). We propose a novel attention-based model called ERNIE-ATT that effectively integrates the diverse levels of semantics acquired by PTMs, thereby enhancing robustness. Experimental evaluations on two challenging datasets showcase the superior performance of our proposed model. It outperforms not only traditional models that do not use PTMs but also exhibits a significant improvement over strong PTM-based models. These findings demonstrate the effectiveness of our approach in enhancing the robustness of question matching/retrieval systems.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SemanticsNatural Language ProcessingHumansNeural Networks, ComputerAttention

Autor:innen und Institutionen

  • College of Computer Science & Information Physics Fusion Intelligent Computing Key Laboratory of the National Ethnic Affairs Commission, South-Central Minzu University, Wuhan, Hubei, China.
  • College of International Business and Economics, Wuhan Textile University, Wuhan, Hubei, China.

ORCID-Profile: Zheng Ye, Linwei Che

Förderung

  • Fundamental Research Funds for the Central Universities of South-Central Minzu University
  • Talent Introduction Program Funds for the Central Universities of South-Central Minzu University

Zitieren

APA

Ye, Z., Che, L., Ge, J., Qin, J., & Liu, J. (2024). Integration of multi-level semantics in PTMs with an attention model for question matching. PLOS ONE. https://doi.org/10.1371/journal.pone.0305772

Vancouver

Ye Z, Che L, Ge J, Qin J, Liu J. Integration of multi-level semantics in PTMs with an attention model for question matching. PLOS ONE. 2024. doi:10.1371/journal.pone.0305772

BibTeX
@article{Ye2024,
  author = {Ye, Zheng and Che, Linwei and Ge, Jun and Qin, Jun and Liu, Jing},
  title = {Integration of multi-level semantics in PTMs with an attention model for question matching},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0305772}
}
Kostenlos starten →