EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Dual experience replay enhanced deep deterministic policy gradient for efficient continuous data sampling

Teh Noranis Mohd Aris, Ningning Chen iD, Norwati Mustapha, Maslina Zolkepli

PLOS One · 11.11.2025 · 20 (11) · e0334411

DOI: 10.1371/journal.pone.0334411 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 11.11.2025. Verfasst von Teh Noranis Mohd Aris und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Universiti Putra Malaysia, UPM, Serdang, Selangor, Malaysia. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

To address the inefficiencies in sample utilization and policy instability in asynchronous distributed reinforcement learning, we propose TPDEB—a dual experience replay framework that integrates prioritized sampling and temporal diversity. While recent distributed RL systems have scaled well, they often suffer from instability and inefficient sampling under network-induced delays and stale policy updates—highlighting a gap in robust learning under asynchronous conditions. TPDEB significantly improves convergence speed and robustness by coordinating dual-buffer updates across distributed agents, offering a scalable solution to real-world continuous control tasks. TPDEB addresses these limitations through two key mechanisms: a trajectory-level prioritized replay buffer that captures temporally coherent high-value experiences, and KL-regularized learning that constrains policy drift across actors. Unlike prior approaches relying on a single experience buffer, TPDEB employs a dual-buffer strategy that combines standard and prioritized replay Buffers. This enables better trade-offs between unbiased sampling and value-driven prioritization, improving learning robustness under asynchronous actor updates. Moreover, TPDEB collects more diverse and redundant experience by scaling parallel actor replicas. Empirical evaluations on MuJoCo continuous control benchmarks demonstrate that TPDEB outperforms baseline distributed algorithms in both convergence speed and final performance, especially under constrained actor–learner bandwidth. Ablation studies validate the contribution of each component, showing that trajectory-level prioritization captures high-quality samples more effectively than step-wise methods, and KL-regularization enhances stability across asynchronous updates. These findings support TPDEB as a practical and scalable solution for distributed reinforcement learning systems.

Autor:innen und Institutionen

  • Department of Computer Science, Universiti Putra Malaysia, UPM, Serdang, Selangor, Malaysia.

ORCID-Profile: Ningning Chen

Zitieren

APA

Mohd Aris, T. N., Chen, N., Mustapha, N., & Zolkepli, M. (2025). Dual experience replay enhanced deep deterministic policy gradient for efficient continuous data sampling. PLOS One, 20(11), e0334411. https://doi.org/10.1371/journal.pone.0334411

Vancouver

Mohd Aris TN, Chen N, Mustapha N, Zolkepli M. Dual experience replay enhanced deep deterministic policy gradient for efficient continuous data sampling. PLOS One. 2025;20(11):e0334411. doi:10.1371/journal.pone.0334411

BibTeX
@article{MohdAris2025,
  author = {Mohd Aris, Teh Noranis and Chen, Ningning and Mustapha, Norwati and Zolkepli, Maslina},
  title = {Dual experience replay enhanced deep deterministic policy gradient for efficient continuous data sampling},
  journal = {PLOS One},
  year = {2025},
  volume = {20},
  number = {11},
  pages = {e0334411},
  doi = {10.1371/journal.pone.0334411}
}
Kostenlos starten →