EUVIMEDEUVIMED entdecken
Crossref

Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization

Zhuo Huang, Muyang Li, Li Shen, Jun Yu, Chen Gong, Bo Han, Tongliang Liu

International Journal of Computer Vision · 2025-01 · 133 (1) · 456-474

DOI: 10.1007/s11263-024-02075-x ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 01.08.2024. Verfasst von Zhuo Huang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 107 Literaturangaben. Gefördert von: Australian Research Council, NSFC Young Scientists Fund, Basic and Applied Basic Research Foundation of Guangdong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Out-of-Distribution (OOD) Generalization aims to learn robust models that generalize well to various environments without fitting to distribution-specific features. Recent studies based on Lottery Ticket Hypothesis (LTH) address this problem by minimizing the learning target to find some of the parameters that are critical to the task. However, in open-world visual recognition problems, such solutions are suboptimal as the learning task contains severe distribution noises, which can mislead the optimization process. Therefore, apart from finding the task-related parameters (i.e., invariant parameters), we propose Exploring Variant parameters for Invariant Learning (EVIL) which also leverages the distribution knowledge to find the parameters that are sensitive to distribution shift (i.e., variant parameters). Once the variant parameters are left out of invariant learning, a robust subnetwork that is resistant to distribution shift can be found. Additionally, the parameters that are relatively stable across distributions can be considered invariant ones to improve invariant learning. By fully exploring both variant and invariant parameters, our EVIL can effectively identify a robust subnetwork to improve OOD generalization. In extensive experiments on integrated testbed: DomainBed, EVIL can effectively and efficiently enhance many popular methods, such as ERM, IRM, SAM, etc. Our code is available at https://github.com/tmllab/EVIL.

Förderung

  • Australian Research Council
  • NSFC Young Scientists Fund
  • Basic and Applied Basic Research Foundation of Guangdong Province
  • RGC Early Career Scheme
  • NSF of China
  • NSF of Jiangsu Province
  • NSF for Distinguished Young Scholar of Jiangsu Province
  • CAAI-Huawei MindSpore Open Fund

Zitieren

APA

Huang, Z., Li, M., Shen, L., Yu, J., Gong, C., Han, B., & Liu, T. (2025). Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization. International Journal of Computer Vision, 133(1), 456-474. https://doi.org/10.1007/s11263-024-02075-x

Vancouver

Huang Z, Li M, Shen L, Yu J, Gong C, Han B, et al. Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization. International Journal of Computer Vision. 2025;133(1):456-474. doi:10.1007/s11263-024-02075-x

BibTeX
@article{Huang2025,
  author = {Huang, Zhuo and Li, Muyang and Shen, Li and Yu, Jun and Gong, Chen and Han, Bo and Liu, Tongliang},
  title = {Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization},
  journal = {International Journal of Computer Vision},
  year = {2025},
  volume = {133},
  number = {1},
  pages = {456--474},
  doi = {10.1007/s11263-024-02075-x}
}
Kostenlos starten →