Neural Architecture Search Using Covariance Matrix Adaptation Evolution Strategy
Evolutionary Computation · 03.06.2024 · 32 (2) · 177-204
DOI: 10.1162/evco_a_00331 ↗Kurz gefasst
Fachartikel in der Zeitschrift Evolutionary Computation, online erschienen am 03.06.2024. Verfasst von Nilotpal Sinha und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu City, 30010, Taiwan nilotpalsinha.cs06@nycu.edu.tw. Hauptthemen laut MEDLINE: Neural Networks, Computer, Algorithms. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 51 Literaturangaben.
Abstract (Originalsprache)
Evolution-based neural architecture search methods have shown promising results, but they require high computational resources because these methods involve training each candidate architecture from scratch and then evaluating its fitness, which results in long search time. Covariance Matrix Adaptation Evolution Strategy (CMA-ES) has shown promising results in tuning hyperparameters of neural networks but has not been used for neural architecture search. In this work, we propose a framework called CMANAS which applies the faster convergence property of CMA-ES to the deep neural architecture search problem. Instead of training each individual architecture seperately, we used the accuracy of a trained one shot model (OSM) on the validation data as a prediction of the fitness of the architecture, resulting in reduced search time. We also used an architecture-fitness table (AF table) for keeping a record of the already evaluated architecture, thus further reducing the search time. The architectures are modeled using a normal distribution, which is updated using CMA-ES based on the fitness of the sampled population. Experimentally, CMANAS achieves better results than previous evolution-based methods while reducing the search time significantly. The effectiveness of CMANAS is shown on two different search spaces using four datasets: CIFAR-10, CIFAR-100, ImageNet, and ImageNet16-120. All the results show that CMANAS is a viable alternative to previous evolution-based methods and extends the application of CMA-ES to the deep neural architecture search field.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu City, 30010, Taiwan nilotpalsinha.cs06@nycu.edu.tw.
- Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu City, 30010, Taiwan kuanwen@cs.nycu.edu.tw.
Zitieren
APA
Sinha, N., & Chen, K. W. (2024). Neural Architecture Search Using Covariance Matrix Adaptation Evolution Strategy. Evolutionary Computation, 32(2), 177-204. https://doi.org/10.1162/evco_a_00331
Vancouver
Sinha N, Chen KW. Neural Architecture Search Using Covariance Matrix Adaptation Evolution Strategy. Evolutionary Computation. 2024;32(2):177-204. doi:10.1162/evco_a_00331
BibTeX
@article{Sinha2024,
author = {Sinha, Nilotpal and Chen, Kuan-Wen},
title = {Neural Architecture Search Using Covariance Matrix Adaptation Evolution Strategy},
journal = {Evolutionary Computation},
year = {2024},
volume = {32},
number = {2},
pages = {177--204},
doi = {10.1162/evco_a_00331}
}
EUVIMED