LaANIL: ANIL with Look-Ahead Meta-Optimization and Data Parallelism
Electronics · 22.04.2024 · 13 (8) · 1585
DOI: 10.3390/electronics13081585 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 22.04.2024. Verfasst von Vasu Tammisetti und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Infineon Technologies AG (Germany), Spanish Ministry of Economic Affairs and Digital Transformation.
Abstract (Originalsprache)
Meta-few-shot learning algorithms, such as Model-Agnostic Meta-Learning (MAML) and Almost No Inner Loop (ANIL), enable machines to learn complex tasks quickly with limited data and based on previous experience. By maintaining the inner loop head of the neural network, ANIL leads to simpler computations and reduces the complexity of MAML. Despite its benefits, ANIL suffers from issues like accuracy variance, slow initial learning, and overfitting, hardening its adaptation and generalization. This work proposes “Look-Ahead ANIL” (LaANIL), an enhancement to ANIL for better learning. LaANIL reorganizes ANIL’s internal architecture, integrating parallel computing techniques (to process multiple training examples simultaneously across computing units) and incorporating Nesterov momentum (which accelerates convergence by adjusting the learning rate based on past gradient information and extracting informative features for look-ahead gradient computation). These additional features make our model more state-of-the-art capable and better edge-compatible and thus improve few-short learning by enabling models to quickly adapt to new information and tasks. LaANIL’s effectiveness is validated on established meta-few-shot learning datasets, including FC100, CIFAR-FS, Mini-ImageNet, CUBirds-200-2011, and Tiered-ImageNet. The proposed model achieved an increased validation accuracy by 7 ± 0.7% and a variance reduction by 44 ± 4% in two-way two-shot classification as well as increased validation by 5 ± 0.4% and a variance reduction by 18 ± 2% in five-way five-shot classification on the FC100 dataset and similarly performed well on other datasets.
Förderung
- Infineon Technologies AG (Germany)
- Spanish Ministry of Economic Affairs and Digital Transformation
Zitieren
APA
Tammisetti, V., Bierzynski, K., Stettinger, G., Morales-Santos, D. P., Cuellar, M. P., & Molina-Solana, M. (2024). LaANIL: ANIL with Look-Ahead Meta-Optimization and Data Parallelism. Electronics, 13(8), 1585. https://doi.org/10.3390/electronics13081585
Vancouver
Tammisetti V, Bierzynski K, Stettinger G, Morales-Santos DP, Cuellar MP, Molina-Solana M. LaANIL: ANIL with Look-Ahead Meta-Optimization and Data Parallelism. Electronics. 2024;13(8):1585. doi:10.3390/electronics13081585
BibTeX
@article{Tammisetti2024,
author = {Tammisetti, Vasu and Bierzynski, Kay and Stettinger, Georg and Morales-Santos, Diego P. and Cuellar, Manuel Pegalajar and Molina-Solana, Miguel},
title = {LaANIL: ANIL with Look-Ahead Meta-Optimization and Data Parallelism},
journal = {Electronics},
year = {2024},
volume = {13},
number = {8},
pages = {1585},
doi = {10.3390/electronics13081585}
}
EUVIMED