Pre-Masking, Preprocessing Completion, and Anomaly Sensitivity in HDFS Log Classification with TF–IDF and LinearSVC
Applied Sciences · 13.11.2026 · 16 (22) · 11256
DOI: 10.3390/app162211256 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 13.11.2026. Verfasst von Selahattin Koç und 1 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Log preprocessing determines both the representation supplied to a detector and whether the intended observations can be evaluated. We compared Drain3 without external regular-expression pre-masking (E), pre-masked Drain3 (D), and a regex bag-of-words representation (R) on HDFS-v1, comprising 11,175,629 log lines and 575,061 labeled block identifiers. Drain3 processing for E and D preceded a fixed 80/20 stratified block split; only D completed the full unlabeled corpus. TF–IDF and LinearSVC were fitted only on the training blocks. E was terminated after approximately 16 h, with 200,109 templates and 42,879 of 115,013 test block identifiers reached, before complete block-document coverage was established. D and R met the operational block-document coverage criterion and produced predictions for all 115,013 test blocks. D yielded 14 false negatives and 81 false positives, whereas R yielded 84 false negatives and 81 false positives. The anomaly recall was 99.584% for D and 97.506% for R. Among 3368 anomalous test blocks, D uniquely detected 73 and R uniquely detected 3, with 11 shared misses (exact two-sided p = 1.94 × 10−18). In this fixed-split setting, external pre-masking enabled the Drain3 branch to complete corpus processing. Under the study’s transductive representation setting, D achieved higher fixed-threshold anomaly sensitivity than R on the common test partition, with the same marginal false-positive count.
Zitieren
APA
Koç, S., & Tulum, G. (2026). Pre-Masking, Preprocessing Completion, and Anomaly Sensitivity in HDFS Log Classification with TF–IDF and LinearSVC. Applied Sciences, 16(22), 11256. https://doi.org/10.3390/app162211256
Vancouver
Koç S, Tulum G. Pre-Masking, Preprocessing Completion, and Anomaly Sensitivity in HDFS Log Classification with TF–IDF and LinearSVC. Applied Sciences. 2026;16(22):11256. doi:10.3390/app162211256
BibTeX
@article{Ko2026,
author = {Koç, Selahattin and Tulum, Gökalp},
title = {Pre-Masking, Preprocessing Completion, and Anomaly Sensitivity in HDFS Log Classification with TF–IDF and LinearSVC},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {22},
pages = {11256},
doi = {10.3390/app162211256}
}
EUVIMED