Sliding Finite Window Codes: Near-Optimality and Q-Learning for Zero-Delay Coding
IEEE Transactions on Information Theory · 2026-06 · 72 (6) · 3910-3926
DOI: 10.1109/tit.2026.3681183 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Information Theory, 2026-06. Verfasst von Liam Cregg und 2 weiteren Autor:innen. Die Arbeit nennt 67 Literaturangaben. Gefördert von: Natural Sciences and Engineering Research Council of Canada.
Abstract (Originalsprache)
We study the problem of zero-delay coding for the transmission of a Markov source over a noisy channel with feedback and present a reinforcement learning solution which is guaranteed to approach optimality. To this end, we formulate the problem as a Markov decision process (MDP) where the state is a probability-measure valued predictor/belief and the actions are quantizer maps. This MDP formulation has been used to show the optimality of certain classes of encoder policies in prior work, but their computation is prohibitively complex due to the uncountable nature of the constructed state space. Based on recent results for partially observed MDPs, we present an approximation of the belief MDP using a sliding finite window of channel outputs and quantizers. Under an appropriate notion of predictor stability, we show that the lowest distortion achievable by such a sliding finite window policy approaches the true lowest distortion as the window length increases. We give sufficient conditions for predictor stability to hold. Finally, we propose a Q-learning algorithm which provably converges to the optimal policy and provide a detailed comparison of the sliding finite window scheme with another approximation scheme which quantizes the belief MDP in a nearest neighbor fashion, as well as other coding schemes from the literature.
Förderung
- Natural Sciences and Engineering Research Council of Canada
Zitieren
APA
Cregg, L., Alajaji, F., & Yüksel, S. (2026). Sliding Finite Window Codes: Near-Optimality and Q-Learning for Zero-Delay Coding. IEEE Transactions on Information Theory, 72(6), 3910-3926. https://doi.org/10.1109/tit.2026.3681183
Vancouver
Cregg L, Alajaji F, Yüksel S. Sliding Finite Window Codes: Near-Optimality and Q-Learning for Zero-Delay Coding. IEEE Transactions on Information Theory. 2026;72(6):3910-3926. doi:10.1109/tit.2026.3681183
BibTeX
@article{Cregg2026,
author = {Cregg, Liam and Alajaji, Fady and Yüksel, Serdar},
title = {Sliding Finite Window Codes: Near-Optimality and Q-Learning for Zero-Delay Coding},
journal = {IEEE Transactions on Information Theory},
year = {2026},
volume = {72},
number = {6},
pages = {3910--3926},
doi = {10.1109/tit.2026.3681183}
}
EUVIMED