A train trajectory optimization method based on the safety reinforcement learning with a relaxed dynamic reward
Discover Applied Sciences · 30.08.2024 · 6 (9)
DOI: 10.1007/s42452-024-06159-8 ↗Kurz gefasst
Fachartikel in der Zeitschrift Discover Applied Sciences, online erschienen am 30.08.2024. Verfasst von Ligang Cheng und 4 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Wuyi University.
Abstract (Originalsprache)
Train trajectory optimization (TTO) is an effective way to address energy consumption in rail transit. Reinforcement learning (RL), an excellent optimization method, has been used to solve TTO problems. Although traditional RL algorithms use penalty functions to restrict the random exploration behavior of agents, they cannot fully guarantee the safety of the process and results. This paper proposes a proximal policy optimization based safety reinforcement learning framework (S-PPO) for the train trajectory optimization, including a safe action rechoosing mechanism (SARM) and a relaxed dynamic reward mechanism (RDRM) combining a relaxed sparse reward and a dynamic dense reward. SARM guarantees that the new states generated by the agent consistently adhere to the environmental security constraints, thereby enhancing sampling efficiency and facilitating algorithm convergence. RDRM makes it easier for agents to obtain successful samples by relaxing time constraints, which also offers a better balance between exploration and exploitation. The experimental results show that S-PPO can significantly improve performance and obtain better train operation trajectories than soft constraint methods, and the convergence process is smoother. Finally, it was demonstrated that S-PPO exhibits good adaptability across various speed limit tracks. Discretize the train operation process based on distance and construct a Markov decision process model. A safety reinforcement learning framework based on PPO is proposed to maintain the learning process within the constraints of boundaries. A relaxed sparse reward which relaxes the constraint of train planned trip time is proposed to enhance the likelihood of agents completing tasks. A dynamic dense reward can balance the contributions of time and energy consumption and offer enhanced feedback. Discretize the train operation process based on distance and construct a Markov decision process model. A safety reinforcement learning framework based on PPO is proposed to maintain the learning process within the constraints of boundaries. A relaxed sparse reward which relaxes the constraint of train planned trip time is proposed to enhance the likelihood of agents completing tasks. A dynamic dense reward can balance the contributions of time and energy consumption and offer enhanced feedback.
Förderung
- Wuyi University
Zitieren
APA
Cheng, L., Cao, J., Yang, X., Wang, W., & Zhou, Z. (2024). A train trajectory optimization method based on the safety reinforcement learning with a relaxed dynamic reward. Discover Applied Sciences, 6(9). https://doi.org/10.1007/s42452-024-06159-8
Vancouver
Cheng L, Cao J, Yang X, Wang W, Zhou Z. A train trajectory optimization method based on the safety reinforcement learning with a relaxed dynamic reward. Discover Applied Sciences. 2024;6(9). doi:10.1007/s42452-024-06159-8
BibTeX
@article{Cheng2024,
author = {Cheng, Ligang and Cao, Jie and Yang, Xiaofeng and Wang, Wenxian and Zhou, Zijian},
title = {A train trajectory optimization method based on the safety reinforcement learning with a relaxed dynamic reward},
journal = {Discover Applied Sciences},
year = {2024},
volume = {6},
number = {9},
doi = {10.1007/s42452-024-06159-8}
}
EUVIMED