CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks
Sensors · 24.05.2023 · 23 (11) · 5042
DOI: 10.3390/s23115042 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 24.05.2023. Verfasst von Gaoqi Tian und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Software, Xinjiang University, Urumqi 830046, China. Hauptthemen laut MEDLINE: Learning, Natural Language Processing. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Social Science Fund of China.
Abstract (Originalsprache)
System logs are a crucial component of system maintainability, as they record the status of the system and essential events for troubleshooting and maintenance when necessary. Therefore, anomaly detection of system logs is crucial. Recent research has focused on extracting semantic information from unstructured log messages for log anomaly detection tasks. Since BERT models work well in natural language processing, this paper proposes an approach called CLDTLog, which introduces contrastive learning and dual-objective tasks in a BERT pre-trained model and performs anomaly detection on system logs through a fully connected layer. This approach does not require log parsing and thus can avoid the uncertainty caused by log parsing. We trained the CLDTLog model on two log datasets (HDFS and BGL) and achieved F1 scores of 0.9971 and 0.9999 on the HDFS and BGL datasets, respectively, which performed better than all known methods. In addition, when using only 1% of the BGL dataset as training data, CLDTLog still achieves an F1 score of 0.9993, showing excellent generalization performance with a significant reduction of the training cost.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Software, Xinjiang University, Urumqi 830046, China.
- College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China.
ORCID-Profile: Gaoqi Tian, Haojie Wu
Förderung
- National Social Science Fund of China
Zitieren
APA
Tian, G., Luktarhan, N., Wu, H., & Shi, Z. (2023). CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks. Sensors, 23(11), 5042. https://doi.org/10.3390/s23115042
Vancouver
Tian G, Luktarhan N, Wu H, Shi Z. CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks. Sensors. 2023;23(11):5042. doi:10.3390/s23115042
BibTeX
@article{Tian2023,
author = {Tian, Gaoqi and Luktarhan, Nurbol and Wu, Haojie and Shi, Zhaolei},
title = {CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks},
journal = {Sensors},
year = {2023},
volume = {23},
number = {11},
pages = {5042},
doi = {10.3390/s23115042}
}
EUVIMED