EUVIMEDEUVIMED entdecken
Crossref

Dual-Reward Reinforcement Learning with Intrinsic Exploration Mechanisms for Real-Time Reservoir Management

Guojing Xin, Kai Zhang, Haochen Wang, Zifeng Sun, Liming Zhang, Piyang Liu, Yang Wang, Jun Yao

SPE Journal · 13.08.2025 · 30 (08) · 4501-4517

DOI: 10.2118/228320-pa ↗

Kurz gefasst

Fachartikel in der Zeitschrift SPE Journal, online erschienen am 18.06.2025. Verfasst von Guojing Xin und 7 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 53 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the process of oil and gas reservoir development, the natural heterogeneity of the reservoir makes the development performance extremely sensitive to the matching of the injection-production well control scheme. Reasonable production optimization is the key to achieving balanced displacement and efficient development. Due to the complexity of oil and gas reservoirs and the large number of variables, traditional production optimization methods often face problems, such as complex and time-consuming calculations, hindering their ability to obtain global optimal solutions. Reinforcement learning (RL), with its intrinsic strengths in sequential decision-making and high-dimensional problem solving, offers a promising alternative for production optimization. Nevertheless, existing RL approaches exhibit limited exploration capabilities in complex reservoir environments, often becoming trapped in local optima. Additionally, they typically fail to incorporate well pattern information effectively, overlooking the location information and the connectivity between injection and production wells. To address these challenges, we propose a dual-reward coupled RL production optimization algorithm based on a global perspective. Specifically, the production optimization problem will be modeled as a Markov decision process. Through the intrinsic reward mechanism, the agent is encouraged to conduct deeper exploration to overcome the optimization limitations in complex environments, and the well network production information is integrated into the model in combination with the graph neural network (GNN) to enhance the modeling ability of the dynamic characteristics of the reservoir. In this way, the agent can effectively avoid relying only on local objectives for optimization and then achieve a more balanced injection and production strategy. The proposed method adaptively learns by engaging in intrinsic-extrinsic directional interactions with the uncertain reservoir environment, effectively leveraging accumulated well control experience in a manner that is similar to real-world field operating mode. Simulation results derived from two reservoir models demonstrate that, in comparison with other optimization methods, the proposed approach achieves higher net present value (NPV) and exhibits outstanding performance in oil displacement.

Zitieren

APA

Xin, G., Zhang, K., Wang, H., Sun, Z., Zhang, L., Liu, P., Wang, Y., & Yao, J. (2025). Dual-Reward Reinforcement Learning with Intrinsic Exploration Mechanisms for Real-Time Reservoir Management. SPE Journal, 30(08), 4501-4517. https://doi.org/10.2118/228320-pa

Vancouver

Xin G, Zhang K, Wang H, Sun Z, Zhang L, Liu P, et al. Dual-Reward Reinforcement Learning with Intrinsic Exploration Mechanisms for Real-Time Reservoir Management. SPE Journal. 2025;30(08):4501-4517. doi:10.2118/228320-pa

BibTeX
@article{Xin2025,
  author = {Xin, Guojing and Zhang, Kai and Wang, Haochen and Sun, Zifeng and Zhang, Liming and Liu, Piyang and Wang, Yang and Yao, Jun},
  title = {Dual-Reward Reinforcement Learning with Intrinsic Exploration Mechanisms for Real-Time Reservoir Management},
  journal = {SPE Journal},
  year = {2025},
  volume = {30},
  number = {08},
  pages = {4501--4517},
  doi = {10.2118/228320-pa}
}
Kostenlos starten →