$\beta $-DARTS++: Bi-Level Regularization for Proxy-Robust Differentiable Architecture Search
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-02 · 48 (2) · 1407-1423
DOI: 10.1109/tpami.2025.3616249 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-02. Verfasst von Peng Ye und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 60 Literaturangaben. Gefördert von: National Key Research and Development Program of China, Natural Science Foundation of Shanghai Municipality, Shanghai Science and Technology Commission Explorer Program.
Abstract (Originalsprache)
Neural Architecture Search (NAS) has attracted increasing attention in recent years because of its capability to design neural networks automatically. Among them, differential NAS approaches such as DARTS, have gained popularity for search efficiency. However, they still suffer from three main issues, that are, the weak stability due to the performance collapse, the poor generalization ability of the searched architectures, and the inferior robustness to different kinds of proxies (i.e., computationally reduced search configurations). To solve the search stability and searched architecture's generalization problems, a simple-but-effective regularization method, termed as Beta-Decay, is proposed to regularize the DARTS-based NAS searching process (referred as $\beta$β-DARTS). Specifically, Beta-Decay regularization can impose constraints to keep the value and variance of activated architecture parameters from being too large, thereby ensuring fair competition among architecture parameters and making the supernet less sensitive to the impact of input on the operation set. In-depth theoretical analyses on how it works and why it works are provided, and comprehensive experiments on a variety of search spaces and datasets validate that Beta-Decay regularization can help to stabilize the searching process and make the searched network more transferable across different datasets. To address the proxy robustness problem, we first benchmark differentiable NAS methods under a wide range of proxy data, proxy channels, proxy layers, and proxy epochs, since the robustness of NAS under different kinds of proxies has not been explored before. We then conclude some interesting findings and find that $\beta$β-DARTS always achieves the best result among all compared NAS methods under almost all proxy settings. We further introduce the novel flooding regularization to the weight optimization of $\beta$β-DARTS (termed as Bi-level regularization), and experimentally and theoretically verify its effectiveness for improving the proxy robustness of differentiable NAS.
Förderung
- National Key Research and Development Program of China
- Natural Science Foundation of Shanghai Municipality
- Shanghai Science and Technology Commission Explorer Program
- Shanghai Municipal Science and Technology Major
Zitieren
APA
Ye, P., He, T., Li, B., Chen, T., Bai, L., & Ouyang, W. (2026). $\beta $-DARTS++: Bi-Level Regularization for Proxy-Robust Differentiable Architecture Search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(2), 1407-1423. https://doi.org/10.1109/tpami.2025.3616249
Vancouver
Ye P, He T, Li B, Chen T, Bai L, Ouyang W. $\beta $-DARTS++: Bi-Level Regularization for Proxy-Robust Differentiable Architecture Search. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(2):1407-1423. doi:10.1109/tpami.2025.3616249
BibTeX
@article{Ye2026,
author = {Ye, Peng and He, Tong and Li, Baopu and Chen, Tao and Bai, Lei and Ouyang, Wanli},
title = {$\beta $-DARTS++: Bi-Level Regularization for Proxy-Robust Differentiable Architecture Search},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {2},
pages = {1407--1423},
doi = {10.1109/tpami.2025.3616249}
}
EUVIMED