Understanding and Enhancing Differentiable Architecture Search from Information Bottleneck Perspective
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (27) · 22500-22508
DOI: 10.1609/aaai.v40i27.39409 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Haidong Kang und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Performance collapse is an intractable issue of Differentiable Architecture Search (DAS), where severe performance degradation of DAS happens when it trains on different search spaces or datasets. We theoretically analyze the issue from the information bottleneck (IB) perspective, and disclose that a solution to overcome this problem is to seek the bifurcation point of IB tradeoff between compression and prediction of the supernet. To this end, we propose a simple yet highly effective method, namely, Batch Entropy-decay Regularization (BER), to guide the learning of DAS, which restricts compression in DAS by imposing a penalty on the architecture parameters. Comprehensive theoretical analyses demonstrate that BER is able to completely resolve DAS's performance collapse issue. Compared with a number of state-of-the-art DAS variants, BER shows its overwhelmingly better performance on 7 search spaces (i.e., NAS-Bench-201, DARTS, S1-S4, MobileNet-like) and 5 popular datasets (i.e., CIFAR-10, CIFAR-100, ImageNet1k, PASCAL VOC 2007, and MS COCO 2017).
Zitieren
APA
Kang, H., Ma, L., Chen, P., He, Q., & Yi, B. (2026). Understanding and Enhancing Differentiable Architecture Search from Information Bottleneck Perspective. Proceedings of the AAAI Conference on Artificial Intelligence, 40(27), 22500-22508. https://doi.org/10.1609/aaai.v40i27.39409
Vancouver
Kang H, Ma L, Chen P, He Q, Yi B. Understanding and Enhancing Differentiable Architecture Search from Information Bottleneck Perspective. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(27):22500-22508. doi:10.1609/aaai.v40i27.39409
BibTeX
@article{Kang2026,
author = {Kang, Haidong and Ma, Lianbo and Chen, Pengjun and He, Qiang and Yi, Bo},
title = {Understanding and Enhancing Differentiable Architecture Search from Information Bottleneck Perspective},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {27},
pages = {22500--22508},
doi = {10.1609/aaai.v40i27.39409}
}
EUVIMED