EUVIMEDEUVIMED entdecken
Crossref

RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method

Alireza Kavianifar, Saeed Jalili

ACM Transactions on Autonomous and Adaptive Systems · 30.09.2026 · 21 (3) · 1-30

DOI: 10.1145/3779424 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Autonomous and Adaptive Systems, online erschienen am 10.06.2026. Verfasst von Alireza Kavianifar und 1 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 53 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Designing software systems within uncertain and evolving operational contexts is challenging, as pervasive uncertainty can compromise system objectives. Self-adaptive systems address this by dynamically adjusting configurations when objectives are at risk. However, exhaustive exploration of large configuration spaces is time-consuming and computationally expensive. Existing machine learning, search-based, and reinforcement learning approaches often degrade under changing conditions. To overcome these limitations, this article introduces an architecture based on the MAPE-K loop integrated with a Deep Reinforcement Learning (DRL) module enhanced by a novel reward-shaping mechanism. The proposed Reward-Shaped Deep Reinforcement Learning (RS-DRL) method reshapes rewards during experience replay, improving generalization, convergence, and adaptability across dynamic environments. Experiments on IoT case studies (DeltaIoTv1/v2) show that RS-DRL achieves an asymptotic multi-objective reward of \(0.9905\pm 0.0268\) compared to \(0.4559\pm 0.2233\) for the \(\epsilon\) -greedy DQN approach, and near-optimal per-objective asymptotic results (e.g., packet loss \(0.9850\pm 0.0071\) , latency \(0.9964\pm 0.0107\) ). Under the TT goal, RS-DRL reduces packet loss by up to \(22.23\%\) and latency by up to \(39.07\%\) compared to the best competing method ( DLASER+ ), while also consistently outperforming the exhaustive-search Reference baseline that analyzes the full adaptation space. All comparisons are based on 30 independent runs, with 25 of 28 baseline comparisons statistically significant (many with p < 0.001). These results demonstrate that RS-DRL offers a robust, efficient, and adaptive optimization strategy for self-adaptive systems operating under uncertainty and dynamic conditions.

Zitieren

APA

Kavianifar, A., & Jalili, S. (2026). RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method. ACM Transactions on Autonomous and Adaptive Systems, 21(3), 1-30. https://doi.org/10.1145/3779424

Vancouver

Kavianifar A, Jalili S. RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method. ACM Transactions on Autonomous and Adaptive Systems. 2026;21(3):1-30. doi:10.1145/3779424

BibTeX
@article{Kavianifar2026,
  author = {Kavianifar, Alireza and Jalili, Saeed},
  title = {RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method},
  journal = {ACM Transactions on Autonomous and Adaptive Systems},
  year = {2026},
  volume = {21},
  number = {3},
  pages = {1--30},
  doi = {10.1145/3779424}
}
Kostenlos starten →