Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization
Environmental Science & Technology · 21.11.2023 · 57 (46) · 18382-18390
DOI: 10.1021/acs.est.3c00353 ↗Kurz gefasst
Fachartikel in der Zeitschrift Environmental Science & Technology, online erschienen am 05.07.2023. Verfasst von Henry C Croll und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Civil, Construction, and Environmental Engineering, Iowa State University, Ames, Iowa 50011, United States. Hauptthemen laut MEDLINE: Sewage, Water Purification. Die Arbeit wurde bisher 63-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: Iowa State University.
Abstract (Originalsprache)
Treatment of wastewater using activated sludge relies on several complex, nonlinear processes. While activated sludge systems can provide high levels of treatment, including nutrient removal, operating these systems is often challenging and energy intensive. Significant research investment has been made in recent years into improving control optimization of such systems, through both domain knowledge and, more recently, machine learning. This study leverages a novel interface between a common process modeling software and a Python reinforcement learning environment to evaluate four common reinforcement learning algorithms for their ability to minimize treatment energy use while maintaining effluent compliance within the Benchmark Simulation Model No. 1 (BSM1) simulation. Three of the algorithms tested, deep Q-learning, proximal policy optimization, and synchronous advantage actor critic, generally performed poorly over the scenarios tested in this study. In contrast, the twin delayed deep deterministic policy gradient (TD3) algorithm consistently produced a high level of control optimization while maintaining the treatment requirements. Under the best selection of state observation features, TD3 control optimization reduced aeration and pumping energy requirements by 14.3% compared to the BSM1 benchmark control, outperforming the advanced domain-based control strategy of ammonia-based aeration control, although future work is necessary to improve robustness of RL implementation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Substanzen
Autor:innen und Institutionen
- Department of Civil, Construction, and Environmental Engineering, Iowa State University, Ames, Iowa 50011, United States.
- Department of Mechanical Engineering, Iowa State University, Ames, Iowa 50011, United States.
ORCID-Profile: Henry C Croll, Kaoru Ikuma
Förderung
- Iowa State University
Zitieren
APA
Croll, H. C., Ikuma, K., Ong, S. K., & Sarkar, S. (2023). Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization. Environmental Science & Technology, 57(46), 18382-18390. https://doi.org/10.1021/acs.est.3c00353
Vancouver
Croll HC, Ikuma K, Ong SK, Sarkar S. Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization. Environmental Science & Technology. 2023;57(46):18382-18390. doi:10.1021/acs.est.3c00353
BibTeX
@article{Croll2023,
author = {Croll, Henry C and Ikuma, Kaoru and Ong, Say Kee and Sarkar, Soumik},
title = {Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization},
journal = {Environmental Science & Technology},
year = {2023},
volume = {57},
number = {46},
pages = {18382--18390},
doi = {10.1021/acs.est.3c00353}
}
EUVIMED