EUVIMEDEUVIMED entdecken
Crossref

LPPG-RL: Lexicographically Projected Policy Gradient Reinforcement Learning with Subproblem Exploration

Ruiyu Qiu, Rui Wang, Guanghui Yang, Xiang Li, Zhijiang Shao

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25009-25017

DOI: 10.1609/aaai.v40i30.39689 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Ruiyu Qiu und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Lexicographic multi-objective problems, which consist of multiple conflicting subtasks with explicit priorities, are common in real-world applications. Despite the advantages of Reinforcement Learning (RL) in single tasks, extending conventional RL methods to prioritized multiple objectives remains challenging. In particular, traditional Safe RL and Multi-Objective RL (MORL) methods have difficulty enforcing priority orderings efficiently. Therefore, Lexicographic Multi-Objective RL (LMORL) methods have been developed to address these challenges. However, existing LMORL methods either rely on heuristic threshold tuning with prior knowledge or are restricted to discrete domains. To overcome these limitations, we propose Lexicographically Projected Policy Gradient RL (LPPG-RL), a novel LMORL framework which leverages sequential gradient projections to identify feasible policy update directions, thereby enabling LPPG-RL broadly compatible with all policy gradient algorithms in continuous spaces. LPPG-RL reformulates the projection step as an optimization problem, and utilizes Dykstra's projection rather than generic solvers to deliver great speedups, especially for small- to medium-scale instances. In addition, LPPG-RL introduces Subproblem Exploration (SE) to prevent gradient vanishing, accelerate convergence and enhance stability. We provide theoretical guarantees for convergence and establish a lower bound on policy improvement. Finally, through extensive experiments in a 2D navigation environment, we demonstrate the effectiveness of LPPG-RL, showing that it outperforms existing state-of-the-art continuous LMORL methods.

Zitieren

APA

Qiu, R., Wang, R., Yang, G., Li, X., & Shao, Z. (2026). LPPG-RL: Lexicographically Projected Policy Gradient Reinforcement Learning with Subproblem Exploration. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25009-25017. https://doi.org/10.1609/aaai.v40i30.39689

Vancouver

Qiu R, Wang R, Yang G, Li X, Shao Z. LPPG-RL: Lexicographically Projected Policy Gradient Reinforcement Learning with Subproblem Exploration. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25009-25017. doi:10.1609/aaai.v40i30.39689

BibTeX
@article{Qiu2026,
  author = {Qiu, Ruiyu and Wang, Rui and Yang, Guanghui and Li, Xiang and Shao, Zhijiang},
  title = {LPPG-RL: Lexicographically Projected Policy Gradient Reinforcement Learning with Subproblem Exploration},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {30},
  pages = {25009--25017},
  doi = {10.1609/aaai.v40i30.39689}
}
Kostenlos starten →