When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 9407-9422
DOI: 10.1109/tpami.2024.3417214 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von You-Wei Luo und 1 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 66 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Basic and Applied Basic Research Foundation of Guangdong Province, Science and Technology Program of Guangzhou.
Abstract (Originalsprache)
As a crucial step toward real-world learning scenarios with changing environments, dataset shift theory and invariant representation learning algorithm have been extensively studied to relax the identical distribution assumption in classical learning setting. Among the different assumptions on the essential of shifting distributions, generalized label shift (GLS) is the latest developed one which shows great potential to deal with the complex factors within the shift. In this paper, we aim to explore the limitations of current dataset shift theory and algorithm, and further provide new insights by presenting a comprehensive understanding of GLS. From theoretical aspect, two informative generalization bounds are derived, and the GLS learner are proved to be sufficiently close to optimal target model from the Bayesian perspective. The main results show the insufficiency of invariant representation learning, and prove the sufficiency and necessity of GLS correction for generalization, which provide theoretical supports and innovations for exploring generalizable model under dataset shift. From methodological aspect, we provide a unified view of existing shift correction frameworks, and propose a kernel embedding-based correction algorithm (KECA) to minimize the generalization error and achieve successful knowledge transfer. Both theoretical results and extensive experiment evaluations demonstrate the sufficiency and necessity of GLS correction for addressing dataset shift and the superiority of proposed algorithm.
Förderung
- National Natural Science Foundation of China
- Basic and Applied Basic Research Foundation of Guangdong Province
- Science and Technology Program of Guangzhou
- Guangdong Province Key Laboratory of Computational Science
- Sun Yat-sen University
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Luo, Y. W., & Ren, C. X. (2024). When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 9407-9422. https://doi.org/10.1109/tpami.2024.3417214
Vancouver
Luo YW, Ren CX. When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):9407-9422. doi:10.1109/tpami.2024.3417214
BibTeX
@article{Luo2024,
author = {Luo, You-Wei and Ren, Chuan-Xian},
title = {When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {12},
pages = {9407--9422},
doi = {10.1109/tpami.2024.3417214}
}
EUVIMED