The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics
Entropy · 13.05.2026 · 28 (5) · 551
DOI: 10.3390/e28050551 ↗Kurz gefasst
Fachartikel in der Zeitschrift Entropy, online erschienen am 13.05.2026. Verfasst von Chandrajit Bajaj. Beteiligte Einrichtung: Department of Computer Science and Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin, Austin, TX 78712, USA. Die Arbeit nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Peter O’Donnell Foundation, Jim Holland Foundation.
Abstract (Originalsprache)
Real-world autonomous agents learn under nonstationarity, safety constraints, and finite energetic budgets. We develop a framework for perennial learning—agents that continuously refine their models while provably controlling the cost of forgetting—by unifying three classical pillars: Kolmogorov complexity, which equates scientific discovery with algorithmic compression; Landauer’s principle, which assigns a minimal thermodynamic cost of kBTln2 per erased bit to every irreversible model update; and port-Hamiltonian (PH) dynamics, whose (J−R)∇H decomposition separates zero-cost reversible inference from costly irreversible forgetting by construction. The Maxwell demon analogy is formalized: each learning episode is a Szilard cycle in which information acquisition, belief transport, and memory erasure must balance thermodynamically. The information-distance framework, comprising the normalized information distance (NID) and normalized compression distance (NCD), provides a computable geometry for measuring learning progress and guiding curriculum design. We separate theideal uncomputable regularizer based on prefix complexity from the practical compressor/MDL (minimum description length) surrogate that appears in optimization and prove a calibration lemma linking the two under a mild uniform-accuracy assumption. Under explicit regularity, compact-sublevel, and non-energy-extracting assumptions, we prove a passivity speed limit for curriculum-induced contractions of the effective feasible set. Under local asymptotic normality, we reprove that Fisher information is a local posterior codelength proxy rather than an exact theorem about algorithmic entropy. A conditional sequential information-budget proposition shows that the per-stage sample requirement scales as O˜(Δkt/λ⋆), where Δkt is the number of materially changed model coordinates (not the total model complexity kt); the k3→Δk improvement is conditional on a warm-start assumption and a chosen cold-start baseline. A double-integrator running example with a moving obstacle illustrates the architecture.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science and Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin, Austin, TX 78712, USA.
ORCID-Profile: Chandrajit Bajaj
Förderung
- Peter O’Donnell Foundation
- Jim Holland Foundation
Zitieren
APA
Bajaj, C. (2026). The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics. Entropy, 28(5), 551. https://doi.org/10.3390/e28050551
Vancouver
Bajaj C. The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics. Entropy. 2026;28(5):551. doi:10.3390/e28050551
BibTeX
@article{Bajaj2026,
author = {Bajaj, Chandrajit},
title = {The Physics, Information, and Computation of Perennial Learning: Kolmogorov Complexity, Information Distance, and Port-Hamiltonian Thermodynamics},
journal = {Entropy},
year = {2026},
volume = {28},
number = {5},
pages = {551},
doi = {10.3390/e28050551}
}
EUVIMED