EUVIMEDEUVIMED entdecken
Crossref

Pivotal Auto-Encoder via Self-Normalizing ReLU

Nelson Goldenstein, Jeremias Sulam, Yaniv Romano

IEEE Transactions on Signal Processing · 2024 · 72 · 3201-3212

DOI: 10.1109/tsp.2024.3418971 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Signal Processing, 2024. Verfasst von Nelson Goldenstein und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 48 Literaturangaben. Gefördert von: Israel Science Foundation, NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sparse auto-encoders are useful for extracting low-dimensional representations from high-dimensional data. However, their performance degrades sharply when the input noise at test time differs from the noise employed during training. This limitation hinders the applicability of auto-encoders in real-world scenarios where the level of noise in the input is unpredictable. In this paper, we formalize single hidden layer sparse auto-encoders as a transform learning problem. Leveraging the transform modeling interpretation, we propose an optimization problem that leads to a predictive model invariant to the noise level at test time. In other words, the same pre-trained model is able to generalize to different noise levels. The proposed optimization algorithm, derived from the square root lasso, is translated into a new, computationally efficient auto-encoding architecture. After proving that our new method is invariant to the noise level, we evaluate our approach by training networks using the proposed architecture for denoising tasks. Our experimental results demonstrate that the trained models yield a significant improvement in stability against varying types of noise compared to commonly used architectures.

Förderung

  • Israel Science Foundation
  • NSF

Zitieren

APA

Goldenstein, N., Sulam, J., & Romano, Y. (2024). Pivotal Auto-Encoder via Self-Normalizing ReLU. IEEE Transactions on Signal Processing, 72, 3201-3212. https://doi.org/10.1109/tsp.2024.3418971

Vancouver

Goldenstein N, Sulam J, Romano Y. Pivotal Auto-Encoder via Self-Normalizing ReLU. IEEE Transactions on Signal Processing. 2024;72:3201-3212. doi:10.1109/tsp.2024.3418971

BibTeX
@article{Goldenstein2024,
  author = {Goldenstein, Nelson and Sulam, Jeremias and Romano, Yaniv},
  title = {Pivotal Auto-Encoder via Self-Normalizing ReLU},
  journal = {IEEE Transactions on Signal Processing},
  year = {2024},
  volume = {72},
  pages = {3201--3212},
  doi = {10.1109/tsp.2024.3418971}
}
Kostenlos starten →