PPO-GPR: A Custom Proximal Policy Optimization Tool for Active Reinforcement Learning
ACS Engineering Au · 17.06.2026 · 6 (3) · 446-462
DOI: 10.1021/acsengineeringau.5c00122 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACS Engineering Au, online erschienen am 07.04.2026. Verfasst von Etinosa Osaro und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Chemical and Biomolecular Engineering, University of Notre Dame, Notre Dame, Indiana 46556, United States. Die Arbeit nennt 72 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Division of Chemical, Bioengineering, Environmental, and Transport Systems, University of Notre Dame.
Abstract (Originalsprache)
Efficient data selection is critical in domains where data acquisition is expensive and time-consuming, such as material science. In this work, we introduce a novel active learning framework that integrates proximal policy optimization (PPO) with Gaussian process regression (GPR) to strategically select informative data points and thereby enhance predictive modeling. Leveraging the inherent stability and sample efficiency of PPO, achieved through a clipped surrogate objective, the framework guides data acquisition via a custom-designed Gymnasium environment tailored for GPR. In this environment, the PPO agent dynamically chooses data points based on their potential to improve the GPR’s performance, as measured by the R2 score, while preventing redundancy through an action masking mechanism. We apply the proposed methodology to predict the selectivity of methane (CH4) over higher alkanes in metal–organic frameworks (MOFs), focusing on CuBTC and IRMOF-1. The framework is evaluated using both ternary and quaternary gas mixtures, where the performance of the GPR is assessed through metrics such as R2, mean absolute error (MAE), and root mean squared error (RMSE). Across CuBTC and IRMOF-1 in ternary and quaternary hydrocarbon mixtures, PPO-guided acquisition achieves 77–86% data savings relative to full GCMC grids, typically querying only ∼14–23% of the candidate pool while the clipped-update PPO policy converges stably by focusing selections in the pressure–temperature–composition regions where selectivity changes most rapidly. This work shows the potential of combining advanced reinforcement learning techniques with regression models to accelerate material discovery and optimize gas separation processes.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Chemical and Biomolecular Engineering, University of Notre Dame, Notre Dame, Indiana 46556, United States.
ORCID-Profile: Etinosa Osaro, Yamil J Colón
Förderung
- Division of Chemical, Bioengineering, Environmental, and Transport Systems
- University of Notre Dame
Zitieren
APA
Osaro, E., & Colón, Y. J. (2026). PPO-GPR: A Custom Proximal Policy Optimization Tool for Active Reinforcement Learning. ACS Engineering Au, 6(3), 446-462. https://doi.org/10.1021/acsengineeringau.5c00122
Vancouver
Osaro E, Colón YJ. PPO-GPR: A Custom Proximal Policy Optimization Tool for Active Reinforcement Learning. ACS Engineering Au. 2026;6(3):446-462. doi:10.1021/acsengineeringau.5c00122
BibTeX
@article{Osaro2026,
author = {Osaro, Etinosa and Colón, Yamil J},
title = {PPO-GPR: A Custom Proximal Policy Optimization Tool for Active Reinforcement Learning},
journal = {ACS Engineering Au},
year = {2026},
volume = {6},
number = {3},
pages = {446--462},
doi = {10.1021/acsengineeringau.5c00122}
}
EUVIMED