Learning to Learn Task-Adaptive Hyperparameters for Few-Shot Learning
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-03 · 46 (3) · 1441-1454
DOI: 10.1109/tpami.2023.3261387 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-03. Verfasst von Sungyong Baik und 4 weiteren Autor:innen. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 88 Literaturangaben. Gefördert von: IITP, Artificial Intelligence Graduate School Program, Seoul National University.
Abstract (Originalsprache)
The objective of few-shot learning is to design a system that can adapt to a given task with only few examples while achieving generalization. Model-agnostic meta-learning (MAML), which has recently gained the popularity for its simplicity and flexibility, learns a good initialization for fast adaptation to a task under few-data regime. However, its performance has been relatively limited especially when novel tasks are different from tasks previously seen during training. In this work, instead of searching for a better initialization, we focus on designing a better fast adaptation process. Consequently, we propose a new task-adaptive weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can generate per-step hyperparameters for each given task: learning rate and weight decay coefficients. The experimental results validate that learning a good weight update rule for fast adaptation is the equally important component that has drawn relatively less attention in the recent few-shot learning approaches. Surprisingly, fast adaptation from random initialization with ALFA can already outperform MAML. Furthermore, the proposed weight-update rule is shown to consistently improve the task-adaptation capability of MAML across diverse problem domains: few-shot classification, cross-domain few-shot classification, regression, visual tracking, and video frame interpolation.
Förderung
- IITP
- Artificial Intelligence Graduate School Program
- Seoul National University
- Hanyang University
- NRF
- Ministry of Science and ICT, South Korea
- Hyundai Motor Group
Zitieren
APA
Baik, S., Choi, M., Choi, J., Kim, H., & Lee, K. M. (2024). Learning to Learn Task-Adaptive Hyperparameters for Few-Shot Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(3), 1441-1454. https://doi.org/10.1109/tpami.2023.3261387
Vancouver
Baik S, Choi M, Choi J, Kim H, Lee KM. Learning to Learn Task-Adaptive Hyperparameters for Few-Shot Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(3):1441-1454. doi:10.1109/tpami.2023.3261387
BibTeX
@article{Baik2024,
author = {Baik, Sungyong and Choi, Myungsub and Choi, Janghoon and Kim, Heewon and Lee, Kyoung Mu},
title = {Learning to Learn Task-Adaptive Hyperparameters for Few-Shot Learning},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {3},
pages = {1441--1454},
doi = {10.1109/tpami.2023.3261387}
}
EUVIMED