Q-learning as a feedback control mechanism in deterministic systems
An International Journal of Optimization and Control: Theories & Applications (IJOCTA) · 09.06.2026 · 16 (4) · 1273
DOI: 10.36922/ijocta026190075 ↗Kurz gefasst
Fachartikel in der Zeitschrift An International Journal of Optimization and Control: Theories & Applications (IJOCTA), online erschienen am 09.06.2026. Verfasst von Utti M. Rifanti und 3 weiteren Autor:innen.
Abstract (Originalsprache)
This paper studies tabular Q-learning when implemented as a feedback controller in deterministic discrete-time systems, where the learning process interacts with the system dynamics to generate closed-loop trajectories. Unlike classical convergenceresults, which assume infinite visitation as an external sampling condition, this property arises from the interaction between recurrence of the induced state dynamics and persistent exploration under GLIE-compliant ϵ-greedy policies. In particular, we prove that exploration remains sufficiently persistent along recurrent states, implying infinite visitation of all state-action pairs within the recurrent region. The results provide a dynamical-systems characterization of the sampling condition underlying classical Q-learning convergence and establish a formal connection between reinforcement learning and recurrence analysis in discrete-time systems. Numerical experiments on nonlinear epidemic dynamics and linear quadratic control benchmarks illustrate the resulting closed-loop behavior.
Zitieren
APA
Rifanti, U. M., Aryati, L., Susyanto, N., & Susanto, H. (2026). Q-learning as a feedback control mechanism in deterministic systems. An International Journal of Optimization and Control: Theories & Applications (IJOCTA), 16(4), 1273. https://doi.org/10.36922/ijocta026190075
Vancouver
Rifanti UM, Aryati L, Susyanto N, Susanto H. Q-learning as a feedback control mechanism in deterministic systems. An International Journal of Optimization and Control: Theories & Applications (IJOCTA). 2026;16(4):1273. doi:10.36922/ijocta026190075
BibTeX
@article{Rifanti2026,
author = {Rifanti, Utti M. and Aryati, Lina and Susyanto, Nanang and Susanto, Hadi},
title = {Q-learning as a feedback control mechanism in deterministic systems},
journal = {An International Journal of Optimization and Control: Theories & Applications (IJOCTA)},
year = {2026},
volume = {16},
number = {4},
pages = {1273},
doi = {10.36922/ijocta026190075}
}
EUVIMED