EUVIMEDEUVIMED entdecken
Crossref

Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization

Henry C Croll iD, Kaoru Ikuma iD, Say Kee Ong, Soumik Sarkar

Environmental Science & Technology · 21.11.2023 · 57 (46) · 18382-18390

DOI: 10.1021/acs.est.3c00353 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Environmental Science & Technology, online erschienen am 05.07.2023. Verfasst von Henry C Croll und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Civil, Construction, and Environmental Engineering, Iowa State University, Ames, Iowa 50011, United States. Hauptthemen laut MEDLINE: Sewage, Water Purification. Die Arbeit wurde bisher 63-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: Iowa State University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Treatment of wastewater using activated sludge relies on several complex, nonlinear processes. While activated sludge systems can provide high levels of treatment, including nutrient removal, operating these systems is often challenging and energy intensive. Significant research investment has been made in recent years into improving control optimization of such systems, through both domain knowledge and, more recently, machine learning. This study leverages a novel interface between a common process modeling software and a Python reinforcement learning environment to evaluate four common reinforcement learning algorithms for their ability to minimize treatment energy use while maintaining effluent compliance within the Benchmark Simulation Model No. 1 (BSM1) simulation. Three of the algorithms tested, deep Q-learning, proximal policy optimization, and synchronous advantage actor critic, generally performed poorly over the scenarios tested in this study. In contrast, the twin delayed deep deterministic policy gradient (TD3) algorithm consistently produced a high level of control optimization while maintaining the treatment requirements. Under the best selection of state observation features, TD3 control optimization reduced aeration and pumping energy requirements by 14.3% compared to the BSM1 benchmark control, outperforming the advanced domain-based control strategy of ammonia-based aeration control, although future work is necessary to improve robustness of RL implementation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SewageWater PurificationWaste Disposal, FluidAlgorithmsWastewater

Stichwörter der Autor:innen

BSM1control optimizationenergy efficiencymachine learningoperationstwin delayed deep deterministic policy gradient (TD3)

Substanzen

SewageWastewater

Autor:innen und Institutionen

  • Department of Civil, Construction, and Environmental Engineering, Iowa State University, Ames, Iowa 50011, United States.
  • Department of Mechanical Engineering, Iowa State University, Ames, Iowa 50011, United States.

ORCID-Profile: Henry C Croll, Kaoru Ikuma

Förderung

  • Iowa State University

Zitieren

APA

Croll, H. C., Ikuma, K., Ong, S. K., & Sarkar, S. (2023). Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization. Environmental Science & Technology, 57(46), 18382-18390. https://doi.org/10.1021/acs.est.3c00353

Vancouver

Croll HC, Ikuma K, Ong SK, Sarkar S. Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization. Environmental Science & Technology. 2023;57(46):18382-18390. doi:10.1021/acs.est.3c00353

BibTeX
@article{Croll2023,
  author = {Croll, Henry C and Ikuma, Kaoru and Ong, Say Kee and Sarkar, Soumik},
  title = {Systematic Performance Evaluation of Reinforcement Learning Algorithms Applied to Wastewater Treatment Control Optimization},
  journal = {Environmental Science & Technology},
  year = {2023},
  volume = {57},
  number = {46},
  pages = {18382--18390},
  doi = {10.1021/acs.est.3c00353}
}
Kostenlos starten →