Synthetic Hard Negative Samples for Contrastive Learning
Neural Processing Letters · 15.02.2024 · 56 (1)
DOI: 10.1007/s11063-024-11522-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Processing Letters, online erschienen am 15.02.2024. Verfasst von Hengkui Dong und 2 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Postgraduate Research & Practice Innovation Program of Jiangsu Province, National Natural Science Foundation of China.
Abstract (Originalsprache)
Contrastive learning has emerged as an essential approach in self-supervised visual representation learning. Its main goal is to maximize the similarities between augmented versions of the same image (positive pairs), while minimizing the similarities between different images (negative pairs). Recent studies have demonstrated that harder negative samples, i.e., those that are more challenging to differentiate from the anchor sample perform a more crucial function in contrastive learning. However, many existing contrastive learning methods ignore the role of hard negative samples. In order to provide harder negative samples for the network model more efficiently. This paper proposes a novel feature-level sample sampling method, namely sampling synthetic hard negative samples for contrastive learning (SSCL). Specifically, we generate more and harder negative samples by mixing them through linear combination and ensure their reliability by debiasing. Finally, we execute weighted sampling of these negative samples. Compared to state-of-the-art methods, our method can provide more high-quality negative samples. Experiments show that SSCL improves the classification performance on different image datasets and can be readily integrated into existing methods.
Förderung
- Postgraduate Research & Practice Innovation Program of Jiangsu Province
- National Natural Science Foundation of China
Zitieren
APA
Dong, H., Long, X., & Li, Y. (2024). Synthetic Hard Negative Samples for Contrastive Learning. Neural Processing Letters, 56(1). https://doi.org/10.1007/s11063-024-11522-2
Vancouver
Dong H, Long X, Li Y. Synthetic Hard Negative Samples for Contrastive Learning. Neural Processing Letters. 2024;56(1). doi:10.1007/s11063-024-11522-2
BibTeX
@article{Dong2024,
author = {Dong, Hengkui and Long, Xianzhong and Li, Yun},
title = {Synthetic Hard Negative Samples for Contrastive Learning},
journal = {Neural Processing Letters},
year = {2024},
volume = {56},
number = {1},
doi = {10.1007/s11063-024-11522-2}
}
EUVIMED