Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method
Advances in Multimedia · 23.04.2022 · 2022 · 1-11
DOI: 10.1155/2022/7923262 ↗Kurz gefasst
Fachartikel in der Zeitschrift Advances in Multimedia, erschienen am 23.04.2022. Verfasst von Fei Lan. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Chongqing Science and Technology Commission of China.
Abstract (Originalsprache)
TF-IDF (term frequency-inverse document frequency) is one of the traditional text similarity calculation methods based on statistics. Because TF-IDF does not consider the semantic information of words, it cannot accurately reflect the similarity between texts, and semantic information enhanced methods distinguish between text documents poorly because extended vectors with semantic similar terms aggravate the curse of dimensionality. Aiming at this problem, this paper advances a hybrid with the semantic understanding and TF-IDF to calculate the similarity of texts. Based on term similarity weighting tree (TSWT) data structure and the definition of semantic similarity information from the HowNet, the paper firstly discusses text preprocess and filter process and then utilizes the semantic information of those key terms to calculate similarities of text documents according to the weight of the features whose weight is greater than the given threshold. The experimental results show that the hybrid method is better than the pure TF-IDF and the method of semantic understanding at the aspect of accuracy, recall, and F1-metric by different K-means clustering methods.
Förderung
- Chongqing Science and Technology Commission of China
Zitieren
APA
Lan, F. (2022). Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method. Advances in Multimedia, 2022, 1-11. https://doi.org/10.1155/2022/7923262
Vancouver
Lan F. Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method. Advances in Multimedia. 2022;2022:1-11. doi:10.1155/2022/7923262
BibTeX
@article{Lan2022,
author = {Lan, Fei},
title = {Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method},
journal = {Advances in Multimedia},
year = {2022},
volume = {2022},
pages = {1--11},
doi = {10.1155/2022/7923262}
}
EUVIMED