Try with Simpler - An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection
ACM Transactions on Software Engineering and Methodology · 30.06.2024 · 33 (5) · 1-27
DOI: 10.1145/3644386 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 03.06.2024. Verfasst von Lin Yang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 90 Literaturangaben. Gefördert von: National Natural Science Foundation of China, CCF Young Elite Scientists Sponsorship Program.
Abstract (Originalsprache)
With the rapid development of deep learning (DL), the recent trend of log-based anomaly detection focuses on extracting semantic information from log events (i.e., templates of log messages) and designing more advanced DL models for anomaly detection. Indeed, the effectiveness of log-based anomaly detection can be improved, but these DL-based techniques further suffer from the limitations of more heavy dependency on training data (such as data quality or data labels) and higher costs in time and resources due to the complexity and scale of DL models, which hinder their practical use. On the contrary, the techniques based on traditional machine learning or data mining algorithms are less dependent on training data and more efficient but produce worse effectiveness than DL-based techniques, which is mainly caused by the problem of unseen log events (some log events in incoming log messages are unseen in training data) confirmed by our motivating study. Intuitively, if we can improve the effectiveness of traditional techniques to be comparable with advanced DL-based techniques, then log-based anomaly detection can be more practical. Indeed, an existing study in the other area (i.e., linking questions posted on Stack Overflow) has pointed out that traditional techniques with some optimizations can indeed achieve comparable effectiveness with the state-of-the-art DL-based technique, indicating the feasibility of enhancing traditional log-based anomaly detection techniques to some degree. Inspired by the idea of “try-with-simpler,” we conducted the first empirical study to explore the potential of improving traditional techniques for more practical log-based anomaly detection. In this work, we optimized the traditional unsupervised PCA (Principal Component Analysis) technique by incorporating a lightweight semantic-based log representation in it, called SemPCA , and conducted an extensive study to investigate the potential of SemPCA for more practical log-based anomaly detection. By comparing seven log-based anomaly detection techniques (including four DL-based techniques, two traditional techniques, and SemPCA ) on both public and industrial datasets, our results show that SemPCA achieves comparable effectiveness as advanced supervised/semi-supervised DL-based techniques while being much more stable under insufficient training data and more efficient, demonstrating that the traditional technique can still excel after small but useful adaptation.
Förderung
- National Natural Science Foundation of China
- CCF Young Elite Scientists Sponsorship Program
Zitieren
APA
Yang, L., Chen, J., Gao, S., Gong, Z., Zhang, H., Kang, Y., & Li, H. (2024). Try with Simpler - An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection. ACM Transactions on Software Engineering and Methodology, 33(5), 1-27. https://doi.org/10.1145/3644386
Vancouver
Yang L, Chen J, Gao S, Gong Z, Zhang H, Kang Y, et al. Try with Simpler - An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection. ACM Transactions on Software Engineering and Methodology. 2024;33(5):1-27. doi:10.1145/3644386
BibTeX
@article{Yang2024,
author = {Yang, Lin and Chen, Junjie and Gao, Shutao and Gong, Zhihao and Zhang, Hongyu and Kang, Yue and Li, Huaan},
title = {Try with Simpler - An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2024},
volume = {33},
number = {5},
pages = {1--27},
doi = {10.1145/3644386}
}
EUVIMED