EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Brief Tutorial on Reinforcement Learning: From MDP to DDPG

Tian Zhang, Zhirong Su

Machine Learning and Knowledge Extraction · 17.11.2026 · 8 (11) · 385

DOI: 10.3390/make8110385 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 17.11.2026. Verfasst von Tian Zhang und 1 weiteren Autor:innen. Die Arbeit nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Shandong Provincial Natural Science Foundation, National Natural Science Foundation of China, Visiting Scholar Program for Faculty of Shandong Provincial Undergraduate Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Following the evolution from theoretical foundations to advanced deep learning algorithms, a coherent overview of reinforcement learning (RL) is proposed in this tutorial. We begin with the mathematical formalization of sequential decision-making via Markov decision processes (MDPs). In MDPs, the Bellman equation and the Bellman optimality equation play important roles; they provide the basis for policy evaluation and the fundamental condition for optimal behavior, respectively. The movement from these equations to practical algorithms is explored, starting with model-based dynamic programming and progressing to model-free temporal-difference (TD) learning. As a pivotal model-free algorithm, Q-learning directly implements the Bellman optimality equation through sampling. To handle high-dimensional state spaces, function approximation and deep reinforcement learning emerge, exemplified by deep Q-networks (DQN). Thereafter, actor-critic methods address the challenge of continuous action spaces. As a typical actor-critic scheme, the deep deterministic policy gradient (DDPG) algorithm is illustrated in detail to show how it adapts the principles of optimality to continuous control by maintaining separate actor and critic networks. Finally, the tutorial concludes with a unified perspective, observing the development of RL as a logical progression from defining optimality conditions to developing scalable solution algorithms. Furthermore, future directions are summarized.

Förderung

  • Shandong Provincial Natural Science Foundation
  • National Natural Science Foundation of China
  • Visiting Scholar Program for Faculty of Shandong Provincial Undergraduate Universities

Zitieren

APA

Zhang, T., & Su, Z. (2026). A Brief Tutorial on Reinforcement Learning: From MDP to DDPG. Machine Learning and Knowledge Extraction, 8(11), 385. https://doi.org/10.3390/make8110385

Vancouver

Zhang T, Su Z. A Brief Tutorial on Reinforcement Learning: From MDP to DDPG. Machine Learning and Knowledge Extraction. 2026;8(11):385. doi:10.3390/make8110385

BibTeX
@article{Zhang2026,
  author = {Zhang, Tian and Su, Zhirong},
  title = {A Brief Tutorial on Reinforcement Learning: From MDP to DDPG},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {11},
  pages = {385},
  doi = {10.3390/make8110385}
}
Kostenlos starten →