EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Behavior-Constrained Multi-Agent Proximal Policy Optimization for Cooperative Vehicle Control at Future Intersections

Zhenhai Gao, Hesheng Hao, Fei Gao, Rui Zhao

Automotive Innovation · 2025-11 · 8 (4) · 896-912

DOI: 10.1007/s42154-024-00308-w ↗

Kurz gefasst

Fachartikel in der Zeitschrift Automotive Innovation, online erschienen am 01.10.2025. Verfasst von Zhenhai Gao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Autonomous driving technology faces significant safety challenges, particularly at unsignalized intersections. Centralized cooperative methods have been developed to manage the flow of connected and automated vehicles. However, many existing approaches depend on basic control algorithms, leading to lengthy inference time and suboptimal solutions. Consequently, real-time performance, road resource utilization, and traffic efficiency are compromised. While some studies have integrated reinforcement learning (RL) techniques to address these issues, they often comprise safety due to reward-driven optimization and the oversimplification of traffic scenarios, such as designing specific flow directions. These limitations raise concerns about their real-world applicability and safety. To address these shortcomings, this paper introduces a novel behavior-constrained proximal policy optimization (BCPPO) method for RL-based cooperative vehicle control at intersections. First, the problem is formulated as a multi-agent RL task within a Markov Game (MG) framework. A multi-agent proximal policy optimization (MAPPO) algorithm is proposed to handle the complex cooperative dynamics among multiple agents. The policy network employs a Long Short-Term Memory (LSTM) encoder to capture extensive social interaction information among the agents. Second, intersection control problem is formalized within the MG framework, and a safety-enhanced cooperative vehicle control strategy, BCPPO, is proposed. This method integrates formal safety verification and behavior constraints into the training and deployment of MAPPO to ensure safety and robustness. Finally, extensive simulation experiments are conducted across various intersection scenarios to evaluate the performance of BCPPO against RL-based proximal policy optimization (PPO), the rule-based first-come-first-served (FCFS) method, and the optimal control (OC)-based vehicles-intersection control system (VICS). The results demonstrate that BCPPO achieves a zero-collision rate during deployment and enhances driving comfort by 60.75%, compared to the non-safety-aware PPO method, which has a collision rate of about 13.85%. Furthermore, BCPPO improves traffic efficiency by 16.15% in comparison to FCFS and reduces inference time by a factor of 71.73 relative to the VICS method.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Gao, Z., Hao, H., Gao, F., & Zhao, R. (2025). Behavior-Constrained Multi-Agent Proximal Policy Optimization for Cooperative Vehicle Control at Future Intersections. Automotive Innovation, 8(4), 896-912. https://doi.org/10.1007/s42154-024-00308-w

Vancouver

Gao Z, Hao H, Gao F, Zhao R. Behavior-Constrained Multi-Agent Proximal Policy Optimization for Cooperative Vehicle Control at Future Intersections. Automotive Innovation. 2025;8(4):896-912. doi:10.1007/s42154-024-00308-w

BibTeX
@article{Gao2025,
  author = {Gao, Zhenhai and Hao, Hesheng and Gao, Fei and Zhao, Rui},
  title = {Behavior-Constrained Multi-Agent Proximal Policy Optimization for Cooperative Vehicle Control at Future Intersections},
  journal = {Automotive Innovation},
  year = {2025},
  volume = {8},
  number = {4},
  pages = {896--912},
  doi = {10.1007/s42154-024-00308-w}
}
Kostenlos starten →