EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Retrieval-Grounded HDFS Log Anomaly Detection and Deterministic Failure Narrative Generation

Xinzhuo Sun, Ziliang Samuel Zhong, Qiyou Wu

Journal of Computational Systems and Applications · 04.06.2026 · 3 (1) · 15-30

DOI: 10.64229/j6d7fr94 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Computational Systems and Applications, online erschienen am 04.06.2026. Verfasst von Xinzhuo Sun und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives/Scope: This paper evaluates Hadoop Distributed File System (HDFS) log anomaly detection on the public HDFS_100k structured-log subset derived from LogHub, with emphasis on detection, evidence-grounded explanation, and selective refusal under label ambiguity. Methods, Procedures, and Process: After block-level sessionization, the benchmark contains 7,940 traces and 313 anomalous sessions (3.94%). A reproducible hybrid detector that combines linear discriminative scoring, pattern-memory posteriors, trace statistics, and a calibrated stacking stage was implemented. A retrieval-augmented generation-inspired layer then assembles evidence bundles and renders deterministic failure narratives; no external large language model is used. Results, Observations, and Conclusions: On a fixed 60/20/20 split, the proposed model obtains F1-score (the harmonic mean of precision and recall) = 0.6452, precision-recall area under the curve (PR-AUC) = 0.5440, and receiver operating characteristic area under the curve (ROC-AUC) = 0.7562. Although logistic regression and linear support vector machine baselines reach slightly higher fixed-threshold F1-score values of 0.6596, the proposed model provides the best cross-validation ranking quality, with mean PR-AUC = 0.5318 and mean ROC-AUC = 0.7558. Exact-pattern ambiguity explains 32 of the 33 fixed-test errors. Selective refusal improves F1-score to 0.6742 at 89.8% coverage and to 0.9375 at 23.9% coverage. Novel/Additive Information: The study contributes an ambiguity-aware operational pipeline that integrates scoring, explanation, and abstention, providing a transferable evaluation pattern for data-intensive infrastructure, including storage and monitoring systems used in petroleum-industry digital operations.

Zitieren

APA

Sun, X., Zhong, Z. S., & Wu, Q. (2026). Retrieval-Grounded HDFS Log Anomaly Detection and Deterministic Failure Narrative Generation. Journal of Computational Systems and Applications, 3(1), 15-30. https://doi.org/10.64229/j6d7fr94

Vancouver

Sun X, Zhong ZS, Wu Q. Retrieval-Grounded HDFS Log Anomaly Detection and Deterministic Failure Narrative Generation. Journal of Computational Systems and Applications. 2026;3(1):15-30. doi:10.64229/j6d7fr94

BibTeX
@article{Sun2026,
  author = {Sun, Xinzhuo and Zhong, Ziliang Samuel and Wu, Qiyou},
  title = {Retrieval-Grounded HDFS Log Anomaly Detection and Deterministic Failure Narrative Generation},
  journal = {Journal of Computational Systems and Applications},
  year = {2026},
  volume = {3},
  number = {1},
  pages = {15--30},
  doi = {10.64229/j6d7fr94}
}
Kostenlos starten →