EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics

Chandrajit Bajaj iD

Entropy · 13.05.2026 · 28 (5) · 551

DOI: 10.3390/e28050551 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Entropy, online erschienen am 13.05.2026. Verfasst von Chandrajit Bajaj. Beteiligte Einrichtung: Department of Computer Science and Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin, Austin, TX 78712, USA. Die Arbeit nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Peter O’Donnell Foundation, Jim Holland Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Real-world autonomous agents learn under nonstationarity, safety constraints, and finite energetic budgets. We develop a framework for perennial learning—agents that continuously refine their models while provably controlling the cost of forgetting—by unifying three classical pillars: Kolmogorov complexity, which equates scientific discovery with algorithmic compression; Landauer’s principle, which assigns a minimal thermodynamic cost of kBTln2 per erased bit to every irreversible model update; and port-Hamiltonian (PH) dynamics, whose (J−R)∇H decomposition separates zero-cost reversible inference from costly irreversible forgetting by construction. The Maxwell demon analogy is formalized: each learning episode is a Szilard cycle in which information acquisition, belief transport, and memory erasure must balance thermodynamically. The information-distance framework, comprising the normalized information distance (NID) and normalized compression distance (NCD), provides a computable geometry for measuring learning progress and guiding curriculum design. We separate theideal uncomputable regularizer based on prefix complexity from the practical compressor/MDL (minimum description length) surrogate that appears in optimization and prove a calibration lemma linking the two under a mild uniform-accuracy assumption. Under explicit regularity, compact-sublevel, and non-energy-extracting assumptions, we prove a passivity speed limit for curriculum-induced contractions of the effective feasible set. Under local asymptotic normality, we reprove that Fisher information is a local posterior codelength proxy rather than an exact theorem about algorithmic entropy. A conditional sequential information-budget proposition shows that the per-stage sample requirement scales as O˜(Δkt/λ⋆), where Δkt is the number of materially changed model coordinates (not the total model complexity kt); the k3→Δk improvement is conditional on a warm-start assumption and a chosen cold-start baseline. A double-integrator running example with a moving obstacle illustrates the architecture.

Schlagwörter

Stichwörter der Autor:innen

Differential Policy Optimization (DPO)Kolmogorov complexityPort-Hamiltonian systemscurriculum learninginformation thermodynamicsnormalized compression distanceperennial learning

Autor:innen und Institutionen

  • Department of Computer Science and Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin, Austin, TX 78712, USA.

ORCID-Profile: Chandrajit Bajaj

Förderung

  • Peter O’Donnell Foundation
  • Jim Holland Foundation

Zitieren

APA

Bajaj, C. (2026). The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics. Entropy, 28(5), 551. https://doi.org/10.3390/e28050551

Vancouver

Bajaj C. The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics. Entropy. 2026;28(5):551. doi:10.3390/e28050551

BibTeX
@article{Bajaj2026,
  author = {Bajaj, Chandrajit},
  title = {The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics},
  journal = {Entropy},
  year = {2026},
  volume = {28},
  number = {5},
  pages = {551},
  doi = {10.3390/e28050551}
}
Kostenlos starten →