EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

LSTM-Based Anomaly Detection for Industrial Control Systems on the HAI Testbed: A Comparison of Evaluation Protocols and Metrics

Damián Martínez Giracca, Fabio López Pires

Computers · 2026-11-18 · 15 (11) · 850

DOI: 10.3390/computers15110850 ↗

At a glance

Journal article in Computers, published online on 2026-11-18. Written by Damián Martínez Giracca and 1 co-authors. The full text is freely available (open access, CC BY 4.0).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Published results for process-level anomaly detection on the HIL-based Augmented ICS (HAI) dataset range from below 0.3 to above 0.95. This extended version of a conference paper, which reported an accuracy of 0.9996 for a Long Short-Term Memory (LSTM) detector on HAI 20.07, measures how much of this range is due to the evaluation protocol and metric rather than to the detectors. The same trained models are evaluated changing one choice at a time: input definition, data split, class imbalance treatment, threshold selection, point adjustment and evaluated segment. In the conference setting, a rule that copies the previous label scores as high as the LSTM, and a random split of the samples alone yields near-perfect scores. On HAI 23.05, a test-tuned threshold and point adjustment double the reported score of the same detector and compress the differences among six detectors, and random anomaly scores obtain a higher point-adjusted F1 than all of them. Under deployment-style protocols, the best semi-supervised detectors outperform supervised detectors trained on attacks against a subset of the control loops, but none meet a budget of one false alarm per day on the test data. A checklist of evaluation choices, code and data splits are released.

Cite

APA

Giracca, D. M., & Pires, F. L. (2026). LSTM-Based Anomaly Detection for Industrial Control Systems on the HAI Testbed: A Comparison of Evaluation Protocols and Metrics. Computers, 15(11), 850. https://doi.org/10.3390/computers15110850

Vancouver

Giracca DM, Pires FL. LSTM-Based Anomaly Detection for Industrial Control Systems on the HAI Testbed: A Comparison of Evaluation Protocols and Metrics. Computers. 2026;15(11):850. doi:10.3390/computers15110850

BibTeX
@article{Giracca2026,
  author = {Giracca, Damián Martínez and Pires, Fabio López},
  title = {LSTM-Based Anomaly Detection for Industrial Control Systems on the HAI Testbed: A Comparison of Evaluation Protocols and Metrics},
  journal = {Computers},
  year = {2026},
  volume = {15},
  number = {11},
  pages = {850},
  doi = {10.3390/computers15110850}
}
Start for free →