EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Minimizing Stochastic Complexity with Ridge Regression

Antony Mizzi iD, David M Walker iD, Michael Small iD

Entropy · 30.06.2026 · 28 (7) · 735

DOI: 10.3390/e28070735 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Entropy, online erschienen am 30.06.2026. Verfasst von Antony Mizzi und 2 weiteren Autor:innen. Beteiligte Einrichtung: Complex Systems Group, Department of Mathematics and Statistics, The University of Western Australia, Perth, WA 6009, Australia. Die Arbeit nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Australian Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We derive a penalty strength criterion for ridge regression using stochastic complexity, which is a refined variant of the minimum description length principle. Since stochastic complexity does not typically account for the effect of regularization on complexity, despite its ability to simplify models, we are required to make a slight modification to the underlying coding scheme. Our scheme makes use of a weighted ensemble of regularized model fits rather than a mixture of maximum likelihood estimates. Under this modification, regularization is interpreted as reducing model complexity by constraining flexibility. In the case of ridge regression, the complexity penalty term that we derive can be expressed analytically as the log determinant of the residual operator. We demonstrate the effect of this complexity penalty by fitting a linear readout to a reservoir computer, and by performing benchmark testing on publicly available datasets.

Schlagwörter

Stichwörter der Autor:innen

coding theory and techniquesinformation theorymodel selectionregularization

Autor:innen und Institutionen

  • Complex Systems Group, Department of Mathematics and Statistics, The University of Western Australia, Perth, WA 6009, Australia.

ORCID-Profile: Antony Mizzi, David M Walker, Michael Small

Förderung

  • Australian Research Council

Zitieren

APA

Mizzi, A., Walker, D. M., & Small, M. (2026). Minimizing Stochastic Complexity with Ridge Regression. Entropy, 28(7), 735. https://doi.org/10.3390/e28070735

Vancouver

Mizzi A, Walker DM, Small M. Minimizing Stochastic Complexity with Ridge Regression. Entropy. 2026;28(7):735. doi:10.3390/e28070735

BibTeX
@article{Mizzi2026,
  author = {Mizzi, Antony and Walker, David M and Small, Michael},
  title = {Minimizing Stochastic Complexity with Ridge Regression},
  journal = {Entropy},
  year = {2026},
  volume = {28},
  number = {7},
  pages = {735},
  doi = {10.3390/e28070735}
}
Kostenlos starten →