EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks

Gaoqi Tian iD, Nurbol Luktarhan, Haojie Wu iD, Zhaolei Shi

Sensors · 24.05.2023 · 23 (11) · 5042

DOI: 10.3390/s23115042 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 24.05.2023. Verfasst von Gaoqi Tian und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Software, Xinjiang University, Urumqi 830046, China. Hauptthemen laut MEDLINE: Learning, Natural Language Processing. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Social Science Fund of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

System logs are a crucial component of system maintainability, as they record the status of the system and essential events for troubleshooting and maintenance when necessary. Therefore, anomaly detection of system logs is crucial. Recent research has focused on extracting semantic information from unstructured log messages for log anomaly detection tasks. Since BERT models work well in natural language processing, this paper proposes an approach called CLDTLog, which introduces contrastive learning and dual-objective tasks in a BERT pre-trained model and performs anomaly detection on system logs through a fully connected layer. This approach does not require log parsing and thus can avoid the uncertainty caused by log parsing. We trained the CLDTLog model on two log datasets (HDFS and BGL) and achieved F1 scores of 0.9971 and 0.9999 on the HDFS and BGL datasets, respectively, which performed better than all known methods. In addition, when using only 1% of the BGL dataset as training data, CLDTLog still achieves an F1 score of 0.9993, showing excellent generalization performance with a significant reduction of the training cost.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LearningNatural Language ProcessingSemanticsUncertainty

Stichwörter der Autor:innen

bidirectional encoder representation from transformerscontrastive learningdual objective taskslog anomaly detection

Autor:innen und Institutionen

  • School of Software, Xinjiang University, Urumqi 830046, China.
  • College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China.

ORCID-Profile: Gaoqi Tian, Haojie Wu

Förderung

  • National Social Science Fund of China

Zitieren

APA

Tian, G., Luktarhan, N., Wu, H., & Shi, Z. (2023). CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks. Sensors, 23(11), 5042. https://doi.org/10.3390/s23115042

Vancouver

Tian G, Luktarhan N, Wu H, Shi Z. CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks. Sensors. 2023;23(11):5042. doi:10.3390/s23115042

BibTeX
@article{Tian2023,
  author = {Tian, Gaoqi and Luktarhan, Nurbol and Wu, Haojie and Shi, Zhaolei},
  title = {CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {11},
  pages = {5042},
  doi = {10.3390/s23115042}
}
Kostenlos starten →