Searching to Exploit Memorization Effect in Deep Learning With Noisy Labels
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 7833-7849
DOI: 10.1109/tpami.2024.3394552 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Hansi Yang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 98 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Tsinghua University-Tencent Joint Laboratory for Internet Innovation Technology, Research Grants Council of the Hong Kong Special Administrative Region.
Abstract (Originalsprache)
Sample selection approaches are popular in robust learning from noisy labels. However, how to control the selection process properly so that deep networks can benefit from the memorization effect is a hard problem. In this paper, motivated by the success of automated machine learning (AutoML), we propose to control the selection process by bi-level optimization. Specifically, we parameterize the selection process by exploiting the general patterns of the memorization effect in the upper-level, and then update these parameters using predicting accuracy obtained from model training in the lower-level. We further introduce semi-supervised learning algorithms to utiilize noisy-labeled data as unlabeled data. To solve the bi-level optimization problem efficiently, we consider more information from the validation curvature by the Newton method and cubic regularization method. We provide convergence analysis for both optimization methods. Results show that while both methods can converge to an (approximately) stationary point, the cubic regularization method can find better local optimal than the Newton method with less time. Experiments on both benchmark and real-world data sets demonstrate that the proposed searching method can lead to significant improvements upon existing methods. Compared with existing AutoML approaches, our method is much more efficient on finding a good selection schedule.
Förderung
- National Natural Science Foundation of China
- Tsinghua University-Tencent Joint Laboratory for Internet Innovation Technology
- Research Grants Council of the Hong Kong Special Administrative Region
- Research Grants Council of the Hong Kong Special Administrative Region, China
- Young Scientists Fund of National Natural Science Foundation of China
Zitieren
APA
Yang, H., Yao, Q., Han, B., & Kwok, J. T. (2024). Searching to Exploit Memorization Effect in Deep Learning With Noisy Labels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 7833-7849. https://doi.org/10.1109/tpami.2024.3394552
Vancouver
Yang H, Yao Q, Han B, Kwok JT. Searching to Exploit Memorization Effect in Deep Learning With Noisy Labels. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):7833-7849. doi:10.1109/tpami.2024.3394552
BibTeX
@article{Yang2024,
author = {Yang, Hansi and Yao, Quanming and Han, Bo and Kwok, James T},
title = {Searching to Exploit Memorization Effect in Deep Learning With Noisy Labels},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {12},
pages = {7833--7849},
doi = {10.1109/tpami.2024.3394552}
}
EUVIMED