EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments

Max Weissenbacher, Anastasia Borovykh, Georgios Rigas

Flow, Turbulence and Combustion · 2025-09 · 115 (3) · 1357-1378

DOI: 10.1007/s10494-024-00632-5 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Flow, Turbulence and Combustion, online erschienen am 09.01.2025. Verfasst von Max Weissenbacher und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mathematics, Imperial College London, London, SW7 2AZ UK. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 89 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Engineering and Physical Sciences Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Control of chaotic systems has far-reaching implications in engineering, including fluid-based energy and transport systems, among many other fields. In real-world applications, control algorithms typically operate only with partial information about the system ( partial observability ) due to limited sensing, which leads to sub-optimal performance when compared to the case where a controller has access to the full system state ( full observability ). While it is well-known that the effect of partial observability can be mediated by introducing a memory component, which allows the controller to keep track of the system’s partial state history, the effect of the type of memory on performance in chaotic regimes is poorly understood. In this study we investigate the use of reinforcement learning for controlling chaotic flows using only partial observations. We use the chaotic Kuramoto–Sivashinsky equation with a forcing term as a model system. In contrast to previous studies, we consider the flow in a variety of dynamic regimes, ranging from mildly to strongly chaotic. We evaluate the loss of performance as the number of sensors available to the controller decreases. We then compare two different frameworks to incorporate memory into the controller, one based on recurrent neural networks and another novel mechanism based on transformers. We demonstrate that the attention-based framework robustly outperforms the alternatives in a range of dynamic regimes. In particular, our method yields improved control in highly chaotic environments, suggesting that attention-based mechanisms may be better suited to the control of chaotic systems.

Schlagwörter

Stichwörter der Autor:innen

Active flow controlChaosOptimal controlReinforcement learning

Autor:innen und Institutionen

  • Department of Mathematics, Imperial College London, London, SW7 2AZ UK.
  • Department of Aeronautics, Imperial College London, London, SW7 2AZ UK.

Förderung

  • Engineering and Physical Sciences Research Council

Zitieren

APA

Weissenbacher, M., Borovykh, A., & Rigas, G. (2025). Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments. Flow, Turbulence and Combustion, 115(3), 1357-1378. https://doi.org/10.1007/s10494-024-00632-5

Vancouver

Weissenbacher M, Borovykh A, Rigas G. Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments. Flow, Turbulence and Combustion. 2025;115(3):1357-1378. doi:10.1007/s10494-024-00632-5

BibTeX
@article{Weissenbacher2025,
  author = {Weissenbacher, Max and Borovykh, Anastasia and Rigas, Georgios},
  title = {Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments},
  journal = {Flow, Turbulence and Combustion},
  year = {2025},
  volume = {115},
  number = {3},
  pages = {1357--1378},
  doi = {10.1007/s10494-024-00632-5}
}
Kostenlos starten →