A Survey on Information Bottleneck
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-08 · 46 (8) · 5325-5344
DOI: 10.1109/tpami.2024.3366349 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-08. Verfasst von Shizhe Hu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 123-mal zitiert (Crossref) und nennt 190 Literaturangaben. Gefördert von: National Natural Science Foundation of China, China Postdoctoral Science Foundation.
Abstract (Originalsprache)
This survey is for the remembrance of one of the creators of the information bottleneck theory, Prof. Naftali Tishby, passing away at the age of 68 on August, 2021. Information bottleneck (IB), a novel information theoretic approach for pattern analysis and representation learning, has gained widespread popularity since its birth in 1999. It provides an elegant balance between data compression and information preservation, and improves its prediction or representation ability accordingly. This survey summarizes both the theoretical progress and practical applications on IB over the past 20-plus years, where its basic theory, optimization, extensive models and task-oriented algorithms are systematically explored. Existing IB methods are roughly divided into two parts: traditional and deep IB, where the former contains the IBs optimized by traditional machine learning analysis techniques without involving any neural networks, and the latter includes the IBs involving the interpretation, optimization and improvement of deep neural works (DNNs). Specifically, based on the technique taxonomy, traditional IBs are further classified into three categories: Basic, Informative and Propagating IB; While the deep IBs, based on the taxonomy of problem settings, contain Debate: Understanding DNNs with IB, Optimizing DNNs Using IB, and DNN-based IB methods. Furthermore, some potential issues deserving future research are discussed. This survey attempts to draw a more complete picture of IB, from which the subsequent studies can benefit.
Förderung
- National Natural Science Foundation of China
- China Postdoctoral Science Foundation
Zitieren
APA
Hu, S., Lou, Z., Yan, X., & Ye, Y. (2024). A Survey on Information Bottleneck. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8), 5325-5344. https://doi.org/10.1109/tpami.2024.3366349
Vancouver
Hu S, Lou Z, Yan X, Ye Y. A Survey on Information Bottleneck. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(8):5325-5344. doi:10.1109/tpami.2024.3366349
BibTeX
@article{Hu2024,
author = {Hu, Shizhe and Lou, Zhengzheng and Yan, Xiaoqiang and Ye, Yangdong},
title = {A Survey on Information Bottleneck},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {8},
pages = {5325--5344},
doi = {10.1109/tpami.2024.3366349}
}
EUVIMED