Automated Log Intelligence System using Machine Learning Techniques
The American Journal of Applied Sciences · 16.05.2026 · 8 (05) · 84-93
DOI: 10.37547/tajas/warm-09 ↗Kurz gefasst
Fachartikel in der Zeitschrift The American Journal of Applied Sciences, online erschienen am 16.05.2026. Verfasst von Ananya Srivastava und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Every application, server, and network device generate logs whenever it performs an action, making these logs a valuable source of information for determining whether a system is in a normal state or has any fault that can cause failure. In large distributed systems, the volume of generated logs is extremely high, often reaching millions of log entries on a daily basis, which makes the manual analysis of these logs almost impossible. Traditional rule-based log management methods require engineers to anticipate all possible faults and their corresponding alerts, making them ineffective for detecting new types of anomalies according to the changing times. This paper presents a machine learning based log intelligence system that processes raw, unstructured log data and performs anomaly detection through a complete pipeline of log parsing, preprocessing, and finally model evaluation. The dataset used is collected from the LogHub repository and is called the HDFS dataset. The HDFS dataset contains about 11 million logs. In this project we have used 3 approaches for log analysis: Isolation Forest (unsupervised algorithm), Random Forest (supervised algorithm), and LSTM (deep learning). The model is trained on each of the methods listed above and then is evaluated using the metrics of precision, recall, accuracy and F1 score. The result after evaluation shows that random forest achieves the highest F1 score of 0.97, then comes LSTM with the score of 0.89 and Isolation Forest a score of 0.79. The results achieved clearly demonstrate that machine learning and deep learning can be used to detect log anomalies in real world data for IT operations.
Zitieren
APA
Srivastava, A., Singh, S., & Singh, V. (2026). Automated Log Intelligence System using Machine Learning Techniques. The American Journal of Applied Sciences, 8(05), 84-93. https://doi.org/10.37547/tajas/warm-09
Vancouver
Srivastava A, Singh S, Singh V. Automated Log Intelligence System using Machine Learning Techniques. The American Journal of Applied Sciences. 2026;8(05):84-93. doi:10.37547/tajas/warm-09
BibTeX
@article{Srivastava2026,
author = {Srivastava, Ananya and Singh, Shikha and Singh, Vineet},
title = {Automated Log Intelligence System using Machine Learning Techniques},
journal = {The American Journal of Applied Sciences},
year = {2026},
volume = {8},
number = {05},
pages = {84--93},
doi = {10.37547/tajas/warm-09}
}
EUVIMED