Efficient similarity joins for near-duplicate detection
ACM Transactions on Database Systems · 2011-08 · 36 (3) · 1-41
DOI: 10.1145/2000824.2000825 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Database Systems, online erschienen am 26.08.2011. Verfasst von Chuan Xiao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 260-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: Australian Research Council.
Abstract (Originalsprache)
With the increasing amount of data and the need to integrate data from multiple data sources, one of the challenging issues is to identify near-duplicate records efficiently. In this article, we focus on efficient algorithms to find a pair of records such that their similarities are no less than a given threshold. Several existing algorithms rely on the prefix filtering principle to avoid computing similarity values for all possible pairs of records. We propose new filtering techniques by exploiting the token ordering information; they are integrated into the existing methods and drastically reduce the candidate sizes and hence improve the efficiency. We have also studied the implementation of our proposed algorithm in stand-alone and RDBMS-based settings. Experimental results show our proposed algorithms can outperform previous algorithms on several real datasets.
Förderung
- Australian Research Council
Zitieren
APA
Xiao, C., Wang, W., Lin, X., Yu, J. X., & Wang, G. (2011). Efficient similarity joins for near-duplicate detection. ACM Transactions on Database Systems, 36(3), 1-41. https://doi.org/10.1145/2000824.2000825
Vancouver
Xiao C, Wang W, Lin X, Yu JX, Wang G. Efficient similarity joins for near-duplicate detection. ACM Transactions on Database Systems. 2011;36(3):1-41. doi:10.1145/2000824.2000825
BibTeX
@article{Xiao2011,
author = {Xiao, Chuan and Wang, Wei and Lin, Xuemin and Yu, Jeffrey Xu and Wang, Guoren},
title = {Efficient similarity joins for near-duplicate detection},
journal = {ACM Transactions on Database Systems},
year = {2011},
volume = {36},
number = {3},
pages = {1--41},
doi = {10.1145/2000824.2000825}
}
EUVIMED