Integration of multi-level semantics in PTMs with an attention model for question matching
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0305772 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Zheng Ye und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Computer Science & Information Physics Fusion Intelligent Computing Key Laboratory of the National Ethnic Affairs Commission, South-Centra…. Hauptthemen laut MEDLINE: Semantics, Natural Language Processing. Die Arbeit nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundamental Research Funds for the Central Universities of South-Central Minzu University, Talent Introduction Program Funds for the Central Universities of South-Central Minzu University.
Abstract (Originalsprache)
The task of question matching/retrieval focuses on determining whether two questions are semantically equivalent. It has garnered significant attention in the field of natural language processing (NLP) due to its commercial value. While neural network models have made great strides and achieved human-level accuracy, they still face challenges when handling complex scenarios. In this paper, we delve into the utilization of different specializations encoded in different layers of large-scale pre-trained language models (PTMs). We propose a novel attention-based model called ERNIE-ATT that effectively integrates the diverse levels of semantics acquired by PTMs, thereby enhancing robustness. Experimental evaluations on two challenging datasets showcase the superior performance of our proposed model. It outperforms not only traditional models that do not use PTMs but also exhibits a significant improvement over strong PTM-based models. These findings demonstrate the effectiveness of our approach in enhancing the robustness of question matching/retrieval systems.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- College of Computer Science & Information Physics Fusion Intelligent Computing Key Laboratory of the National Ethnic Affairs Commission, South-Central Minzu University, Wuhan, Hubei, China.
- College of International Business and Economics, Wuhan Textile University, Wuhan, Hubei, China.
ORCID-Profile: Zheng Ye, Linwei Che
Förderung
- Fundamental Research Funds for the Central Universities of South-Central Minzu University
- Talent Introduction Program Funds for the Central Universities of South-Central Minzu University
Zitieren
APA
Ye, Z., Che, L., Ge, J., Qin, J., & Liu, J. (2024). Integration of multi-level semantics in PTMs with an attention model for question matching. PLOS ONE. https://doi.org/10.1371/journal.pone.0305772
Vancouver
Ye Z, Che L, Ge J, Qin J, Liu J. Integration of multi-level semantics in PTMs with an attention model for question matching. PLOS ONE. 2024. doi:10.1371/journal.pone.0305772
BibTeX
@article{Ye2024,
author = {Ye, Zheng and Che, Linwei and Ge, Jun and Qin, Jun and Liu, Jing},
title = {Integration of multi-level semantics in PTMs with an attention model for question matching},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0305772}
}
EUVIMED