Bridging classical and artificial intelligence-based flow control: An expert-guided deep reinforcement learning framework
AIP Advances · 01.04.2026 · 16 (4)
DOI: 10.1063/5.0322964 ↗Kurz gefasst
Fachartikel in der Zeitschrift AIP Advances, online erschienen am 03.04.2026. Verfasst von Boyu Pan und 8 weiteren Autor:innen. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Deep Reinforcement Learning (DRL) is a powerful paradigm for discovering non-linear control strategies in complex fluid dynamics. However, its application to high-fidelity simulations is often hampered by prohibitive sample inefficiency and the risks of “cold-start” exploration. To overcome these limitations, an Expert-Guided Soft Actor-Critic (EG-SAC) framework is proposed, which synergistically integrates prior knowledge from a classical, particle swarm optimization-optimized proportional-integral-derivative controller into the DRL agent. The framework employs a two-stage learning process, beginning with an offline phase where behavioral cloning initializes the policy and expert demonstrations pre-fill the replay buffer and an online fine-tuning stage where a composite loss function, featuring a decaying expert-regularization term, provides continuous guidance to the agent. This framework is applied to active flow control for a circular cylinder at a Reynolds number (Re) of 100. The results demonstrate a threefold advantage: first, the EG-SAC agent achieves immediate operational safety, exhibiting near-expert performance from the first epoch, while the standard SAC agent initially suffers from detrimental, high-drag explorations; second, the sample efficiency is significantly enhanced, with asymptotic convergence accelerated by ∼19%; finally, the converged EG-SAC policy achieves a drag reduction of 7.93%, surpassing the 5.21% achieved by the best linear controller and demonstrating the ability to discover superior non-linear control laws. The superior performance stems from an enhanced base pressure recovery via a further elongated recirculation bubble. This work presents a robust and efficient methodology for applying DRL to fluid mechanics, bridging the gap between the stability of classical control and the high-performance adaptability of data-driven methods.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
Zitieren
APA
Pan, B., Guo, D., Ju, S., Sun, Z., Wang, J., Qiu, W., Ren, S., Lu, W., & Yang, G. (2026). Bridging classical and artificial intelligence-based flow control: An expert-guided deep reinforcement learning framework. AIP Advances, 16(4). https://doi.org/10.1063/5.0322964
Vancouver
Pan B, Guo D, Ju S, Sun Z, Wang J, Qiu W, et al. Bridging classical and artificial intelligence-based flow control: An expert-guided deep reinforcement learning framework. AIP Advances. 2026;16(4). doi:10.1063/5.0322964
BibTeX
@article{Pan2026,
author = {Pan, Boyu and Guo, Dilong and Ju, Shengjun and Sun, Zhenxu and Wang, Junxiang and Qiu, Wei and Ren, Shixuan and Lu, Weishuang and Yang, Guowei},
title = {Bridging classical and artificial intelligence-based flow control: An expert-guided deep reinforcement learning framework},
journal = {AIP Advances},
year = {2026},
volume = {16},
number = {4},
doi = {10.1063/5.0322964}
}
EUVIMED