EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Integrating Model Predictive Control with Deep Reinforcement Learning for Robust Control of Thermal Processes with Long Time Delays

Kevin Marlon Soza Mamani, Alvaro Javier Prado Romo

Processes · 22.05.2025 · 13 (6) · 1627

DOI: 10.3390/pr13061627 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Processes, online erschienen am 22.05.2025. Verfasst von Kevin Marlon Soza Mamani und 1 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Sustainable Engineering Doctoral Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Thermal processes with prolonged and variable delays pose considerable difficulties due to unpredictable system dynamics and external disturbances, often resulting in diminished control effectiveness. This work presents a hybrid control strategy that synthesizes deep reinforcement learning (DRL) strategies with nonlinear model predictive control (NMPC) to improve the robust control performance of a thermal process with a long time delay. In this approach, NMPC cost functions are formulated as learning functions to achieve control objectives in terms of thermal tracking and disturbance rejection, while an actor–critic (AC) reinforcement learning agent dynamically adjusts control actions through an adaptive policy based on the exploration and exploitation of real-time data about the thermal process. Unlike conventional NMPC approaches, the proposed framework removes the need for predefined terminal cost tuning and strict constraint formulations during the control execution at runtime, which are typically required to ensure robust stability. To assess performance, a comparative study was conducted evaluating NMPC against AC-based controllers built upon policy gradient algorithms such as the deep deterministic policy gradient (DDPG) and the twin delayed deep deterministic policy gradient (TD3). The proposed method was experimentally validated using a temperature control laboratory (TCLab) testbed featuring long and varying delays. Results demonstrate that while the NMPC–AC hybrid approach maintains tracking control performance comparable to NMPC, the proposed technique acquires adaptability while tracking and further strengthens robustness in the presence of uncertainties and disturbances under dynamic system conditions. These findings highlight the benefits of integrating DRL with NMPC to enhance reliability in thermal process control and optimize resource efficiency in thermal applications.

Förderung

  • Sustainable Engineering Doctoral Program

Zitieren

APA

Mamani, K. M. S., & Romo, A. J. P. (2025). Integrating Model Predictive Control with Deep Reinforcement Learning for Robust Control of Thermal Processes with Long Time Delays. Processes, 13(6), 1627. https://doi.org/10.3390/pr13061627

Vancouver

Mamani KMS, Romo AJP. Integrating Model Predictive Control with Deep Reinforcement Learning for Robust Control of Thermal Processes with Long Time Delays. Processes. 2025;13(6):1627. doi:10.3390/pr13061627

BibTeX
@article{Mamani2025,
  author = {Mamani, Kevin Marlon Soza and Romo, Alvaro Javier Prado},
  title = {Integrating Model Predictive Control with Deep Reinforcement Learning for Robust Control of Thermal Processes with Long Time Delays},
  journal = {Processes},
  year = {2025},
  volume = {13},
  number = {6},
  pages = {1627},
  doi = {10.3390/pr13061627}
}
Kostenlos starten →