EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method

Fei Lan

Advances in Multimedia · 23.04.2022 · 2022 · 1-11

DOI: 10.1155/2022/7923262 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advances in Multimedia, erschienen am 23.04.2022. Verfasst von Fei Lan. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Chongqing Science and Technology Commission of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

TF-IDF (term frequency-inverse document frequency) is one of the traditional text similarity calculation methods based on statistics. Because TF-IDF does not consider the semantic information of words, it cannot accurately reflect the similarity between texts, and semantic information enhanced methods distinguish between text documents poorly because extended vectors with semantic similar terms aggravate the curse of dimensionality. Aiming at this problem, this paper advances a hybrid with the semantic understanding and TF-IDF to calculate the similarity of texts. Based on term similarity weighting tree (TSWT) data structure and the definition of semantic similarity information from the HowNet, the paper firstly discusses text preprocess and filter process and then utilizes the semantic information of those key terms to calculate similarities of text documents according to the weight of the features whose weight is greater than the given threshold. The experimental results show that the hybrid method is better than the pure TF-IDF and the method of semantic understanding at the aspect of accuracy, recall, and F1-metric by different K-means clustering methods.

Förderung

  • Chongqing Science and Technology Commission of China

Zitieren

APA

Lan, F. (2022). Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method. Advances in Multimedia, 2022, 1-11. https://doi.org/10.1155/2022/7923262

Vancouver

Lan F. Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method. Advances in Multimedia. 2022;2022:1-11. doi:10.1155/2022/7923262

BibTeX
@article{Lan2022,
  author = {Lan, Fei},
  title = {Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method},
  journal = {Advances in Multimedia},
  year = {2022},
  volume = {2022},
  pages = {1--11},
  doi = {10.1155/2022/7923262}
}
Kostenlos starten →