EUVIMEDEUVIMED entdecken
Crossref

Conditional computation in neural networks: Principles and research trends

Simone Scardapane, Alessandro Baiocchi, Alessio Devoto, Valerio Marsocci, Pasquale Minervini, Jary Pomponi

Intelligenza Artificiale · 31.07.2024 · 18 (1) · 175-190

DOI: 10.3233/ia-240035 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Intelligenza Artificiale, erschienen am 31.07.2024. Verfasst von Simone Scardapane und 5 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 115 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article summarizes principles and ideas from the emerging area of applying conditional computation methods to the design of neural networks. In particular, we focus on neural networks that can dynamically activate or de-activate parts of their computational graph conditionally on their input. Examples include the dynamic selection of, e.g., input tokens, layers (or sets of layers), and sub-modules inside each layer (e.g., channels in a convolutional filter). We first provide a general formalism to describe these techniques in an uniform way. Then, we introduce three notable implementations of these principles: mixture-of-experts (MoEs) networks, token selection mechanisms, and early-exit neural networks. The paper aims to provide a tutorial-like introduction to this growing field. To this end, we analyze the benefits of these modular designs in terms of efficiency, explainability, and transfer learning, with a focus on emerging applicative areas ranging from automated scientific discovery to semantic communication.

Zitieren

APA

Scardapane, S., Baiocchi, A., Devoto, A., Marsocci, V., Minervini, P., & Pomponi, J. (2024). Conditional computation in neural networks: Principles and research trends. Intelligenza Artificiale, 18(1), 175-190. https://doi.org/10.3233/ia-240035

Vancouver

Scardapane S, Baiocchi A, Devoto A, Marsocci V, Minervini P, Pomponi J. Conditional computation in neural networks: Principles and research trends. Intelligenza Artificiale. 2024;18(1):175-190. doi:10.3233/ia-240035

BibTeX
@article{Scardapane2024,
  author = {Scardapane, Simone and Baiocchi, Alessandro and Devoto, Alessio and Marsocci, Valerio and Minervini, Pasquale and Pomponi, Jary},
  title = {Conditional computation in neural networks: Principles and research trends},
  journal = {Intelligenza Artificiale},
  year = {2024},
  volume = {18},
  number = {1},
  pages = {175--190},
  doi = {10.3233/ia-240035}
}
Kostenlos starten →