EUVIMEDEUVIMED entdecken
ÜbersichtsarbeitOpen Access · CC BY 4.0

Everything Is Prediction: Modern Machine Learning as Bayesian Inference

Nicholas G Polson, Vadim Sokolov iD, Refik Soyer iD

Entropy · 01.08.2026 · 28 (8) · 869

DOI: 10.3390/e28080869 ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift Entropy, online erschienen am 01.08.2026. Verfasst von Nicholas G Polson und 2 weiteren Autor:innen. Beteiligte Einrichtung: Booth School of Business, University of Chicago, Chicago, IL 60637, USA. Die Arbeit nennt 108 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We argue that the core methods of modern machine learning—conformal prediction, large language models and in-context learning, and generative/diffusion models—are not rivals to Bayesian inference but implementations of it, almost always of its predictive (de Finetti) form rather than its parameter-centric (prior-to-posterior) form. (1)

Background

A quarter-century after Breiman contrasted the “data-modeling” and “algorithmic” cultures of statistics, we revisit that dichotomy and argue that the predictive view dissolves it—both cultures target the one-step-ahead density p(yn+1∣y1:n), differing only in how they compute it. (2)

Methods

We organize the modern toolkit around this predictive object and around amortization—replacing per-dataset inference with a single map learned by simulation—using Generative Bayesian Computation (GBC) as the connective spine. (3)

Results

Conformal prediction, autoregressive language models, prior-data fitted networks, and score-based diffusion are each shown to construct, calibrate, or sample from the predictive object, summarized in a single “Rosetta” table; because all are fit by proper scoring rules—equivalently, by Bregman divergences—the information-theoretic frame is the natural unifier. (4)

Conclusions

The equivalence is exact in idealized limits, asymptotic under exchangeability and its martingale relaxations, and measurably approximate for trained models—a three-grade taxonomy that we make explicit, row by row, and that bounds the thesis: prediction is not attribution, and the predictive view is deliberately silent about causal structures.

Schlagwörter

Stichwörter der Autor:innen

Bregman divergenceconformal predictiondiffusion modelsentropyexchangeabilitygenerative Bayesian computationin-context learningmartingale posteriorsmaximum-entropy reinforcement learningpredictive inferenceproper scoring rulestwo cultures

Autor:innen und Institutionen

  • Booth School of Business, University of Chicago, Chicago, IL 60637, USA.
  • Department of Systems Engineering and Operations Research, George Mason University, Fairfax, VA 22030, USA.
  • Department of Decision Sciences, School of Business, George Washington University, Washington, DC 20052, USA.

ORCID-Profile: Vadim Sokolov, Refik Soyer

Zitieren

APA

Polson, N. G., Sokolov, V., & Soyer, R. (2026). Everything Is Prediction: Modern Machine Learning as Bayesian Inference. Entropy, 28(8), 869. https://doi.org/10.3390/e28080869

Vancouver

Polson NG, Sokolov V, Soyer R. Everything Is Prediction: Modern Machine Learning as Bayesian Inference. Entropy. 2026;28(8):869. doi:10.3390/e28080869

BibTeX
@article{Polson2026,
  author = {Polson, Nicholas G and Sokolov, Vadim and Soyer, Refik},
  title = {Everything Is Prediction: Modern Machine Learning as Bayesian Inference},
  journal = {Entropy},
  year = {2026},
  volume = {28},
  number = {8},
  pages = {869},
  doi = {10.3390/e28080869}
}
Kostenlos starten →