EUVIMEDOdkrijte EUVIMED
Open access · CC BY 4.0

Local Entropy Inversion in Large-Scale AI Systems: Landauer Bounds on Algorithmic Compression

Boris Kriger

IPI Letters · 2026-03-20 · 13-20

DOI: 10.59973/ipil.335 ↗

At a glance

Journal article in IPI Letters, published online on 2026-03-20. Written by Boris Kriger. 24 references. The full text is freely available (open access, CC BY 4.0).

PovzetekBibliografski podatkiCiteRelated studies

Abstract (original language)

We apply Landauer's principle to the training of large language models (LLMs), framing the process as a physically irreversible compression of high-entropy data distributions into low-entropy structured representations stored in model weights. This yields a lower bound on the minimum energy required for AI training, expressed in terms of the information-theoretic compression achieved. Empirical analysis of contemporary AI systems—GPT-3, PaLM, and LLaMA-2—reveals that current implementations operate approximately 10²¹ times above this Landauer limit. We introduce a demon efficiency metric to quantify this gap and examine how it varies across systems and baseline assumptions. We discuss an instructive analogy between LLM training and Maxwell's demon that provides physical intuition for the entropy-reducing character of the training process. We present a sensitivity analysis showing that while the absolute value of the efficiency metric depends on the choice of entropy baseline, the order-of-magnitude gap to the Landauer limit is robust across reasonable choices. These results provide a physical perspective on the energy requirements of artificial intelligence, though we emphasise that the Landauer bound is a direct consequence of well-established thermodynamic principles rather than a new theoretical result.

Cite

APA

Kriger, B. (2026). Local Entropy Inversion in Large-Scale AI Systems: Landauer Bounds on Algorithmic Compression. IPI Letters, 13-20. https://doi.org/10.59973/ipil.335

Vancouver

Kriger B. Local Entropy Inversion in Large-Scale AI Systems: Landauer Bounds on Algorithmic Compression. IPI Letters. 2026:13-20. doi:10.59973/ipil.335

BibTeX
@article{Kriger2026,
  author = {Kriger, Boris},
  title = {Local Entropy Inversion in Large-Scale AI Systems: Landauer Bounds on Algorithmic Compression},
  journal = {IPI Letters},
  year = {2026},
  pages = {13--20},
  doi = {10.59973/ipil.335}
}
Start for free →