EUVIMEDEUVIMED entdecken
Crossref

Joint Optimization of Discrete and Continuous Reservoir Control Strategies Using Recent Experience-Based Hybrid Maximum Entropy Deep Reinforcement Learning

Guojing Xin, Kai Zhang, Zifeng Sun, Liming Zhang, Zhongzheng Wang, Wenjuan Zhang, Piyang Liu, Xia Yan, Yongfei Yang, Hai Sun

SPE Journal · 07.01.2026 · 31 (01) · 520-540

DOI: 10.2118/231184-pa ↗

Kurz gefasst

Fachartikel in der Zeitschrift SPE Journal, online erschienen am 19.11.2025. Verfasst von Guojing Xin und 9 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 71 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Petroleum resources remain the cornerstone of global energy supply, and the joint optimization of discrete and continuous measures is crucial to reservoir development. The coordination of discrete layer-blocking decisions and continuous well-control operations poses significant challenges, where existing methods often convert discrete variables into continuous ones and require reoptimization for different reservoir development scenarios, hindering efficient and accurate optimization. In this paper, we propose a hybrid maximum entropy deep reinforcement learning (DRL) optimization method guided by recent experience (GRE-HMEDRL), which is used for the joint optimization of discrete layer selection and continuous well control parameters. The joint optimization problem is modeled as a Markov decision process (MDP) with a hybrid action space of discrete and continuous actions, integrating discrete and continuous soft actor-critic (SAC) algorithms within a maximum entropy framework. The agent interacts with the reservoir numerical simulator in real time, accumulating historical experiences, and the recent-experience sampling mechanism dynamically adjusts the sampling range, enabling rapid training of a hybrid action policy that maximizes the net present value (NPV) of reservoir development. The hybrid action policy extracts latent state representations of the reservoir through a deep convolutional neural network (CNN) shared by both discrete and continuous action policies, outputting layer blocking and well control variables for joint optimization. Extensive experiments on two 3D reservoir models demonstrate that GRE-HMEDRL consistently outperforms gradient-based, evolutionary algorithms, and existing DRL methods in terms of optimization performance, achieving the highest NPV with greater oil recovery and reduced water production. Compared with baseline methods, GRE-HMEDRL exhibits faster convergence, improved robustness, and higher efficiency. Notably, the trained hybrid action policy demonstrates strong robustness and adaptability in offline deployment across diverse scenarios, including increased regulation frequency and well failures. Without retraining, it maintains near-optimal performance (within a 0.1–4.2% NPV gap) while avoiding the need for thousands of additional reservoir simulations, significantly reducing computational cost and enabling real-time application in practical reservoir management. In this paper, we use well control and layer selection as a representative example, but the GRE-HMEDRL framework is readily applicable to other coupled discrete-continuous optimization problems.

Zitieren

APA

Xin, G., Zhang, K., Sun, Z., Zhang, L., Wang, Z., Zhang, W., Liu, P., Yan, X., Yang, Y., & Sun, H. (2026). Joint Optimization of Discrete and Continuous Reservoir Control Strategies Using Recent Experience-Based Hybrid Maximum Entropy Deep Reinforcement Learning. SPE Journal, 31(01), 520-540. https://doi.org/10.2118/231184-pa

Vancouver

Xin G, Zhang K, Sun Z, Zhang L, Wang Z, Zhang W, et al. Joint Optimization of Discrete and Continuous Reservoir Control Strategies Using Recent Experience-Based Hybrid Maximum Entropy Deep Reinforcement Learning. SPE Journal. 2026;31(01):520-540. doi:10.2118/231184-pa

BibTeX
@article{Xin2026,
  author = {Xin, Guojing and Zhang, Kai and Sun, Zifeng and Zhang, Liming and Wang, Zhongzheng and Zhang, Wenjuan and Liu, Piyang and Yan, Xia and Yang, Yongfei and Sun, Hai},
  title = {Joint Optimization of Discrete and Continuous Reservoir Control Strategies Using Recent Experience-Based Hybrid Maximum Entropy Deep Reinforcement Learning},
  journal = {SPE Journal},
  year = {2026},
  volume = {31},
  number = {01},
  pages = {520--540},
  doi = {10.2118/231184-pa}
}
Kostenlos starten →