Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments
Flow, Turbulence and Combustion · 2025-09 · 115 (3) · 1357-1378
DOI: 10.1007/s10494-024-00632-5 ↗Kurz gefasst
Fachartikel in der Zeitschrift Flow, Turbulence and Combustion, online erschienen am 09.01.2025. Verfasst von Max Weissenbacher und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mathematics, Imperial College London, London, SW7 2AZ UK. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 89 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Engineering and Physical Sciences Research Council.
Abstract (Originalsprache)
Control of chaotic systems has far-reaching implications in engineering, including fluid-based energy and transport systems, among many other fields. In real-world applications, control algorithms typically operate only with partial information about the system ( partial observability ) due to limited sensing, which leads to sub-optimal performance when compared to the case where a controller has access to the full system state ( full observability ). While it is well-known that the effect of partial observability can be mediated by introducing a memory component, which allows the controller to keep track of the system’s partial state history, the effect of the type of memory on performance in chaotic regimes is poorly understood. In this study we investigate the use of reinforcement learning for controlling chaotic flows using only partial observations. We use the chaotic Kuramoto–Sivashinsky equation with a forcing term as a model system. In contrast to previous studies, we consider the flow in a variety of dynamic regimes, ranging from mildly to strongly chaotic. We evaluate the loss of performance as the number of sensors available to the controller decreases. We then compare two different frameworks to incorporate memory into the controller, one based on recurrent neural networks and another novel mechanism based on transformers. We demonstrate that the attention-based framework robustly outperforms the alternatives in a range of dynamic regimes. In particular, our method yields improved control in highly chaotic environments, suggesting that attention-based mechanisms may be better suited to the control of chaotic systems.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Mathematics, Imperial College London, London, SW7 2AZ UK.
- Department of Aeronautics, Imperial College London, London, SW7 2AZ UK.
Förderung
- Engineering and Physical Sciences Research Council
Zitieren
APA
Weissenbacher, M., Borovykh, A., & Rigas, G. (2025). Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments. Flow, Turbulence and Combustion, 115(3), 1357-1378. https://doi.org/10.1007/s10494-024-00632-5
Vancouver
Weissenbacher M, Borovykh A, Rigas G. Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments. Flow, Turbulence and Combustion. 2025;115(3):1357-1378. doi:10.1007/s10494-024-00632-5
BibTeX
@article{Weissenbacher2025,
author = {Weissenbacher, Max and Borovykh, Anastasia and Rigas, Georgios},
title = {Reinforcement Learning of Chaotic Systems Control in Partially Observable Environments},
journal = {Flow, Turbulence and Combustion},
year = {2025},
volume = {115},
number = {3},
pages = {1357--1378},
doi = {10.1007/s10494-024-00632-5}
}
EUVIMED