EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

OpsLLM for Cloud Incident Triage: Bilingual RAG-Based Root Cause Analysis and Alert Summarization for AI Infrastructure Operations

Ge Liu, Chenyu Li, Eric Zhang

Journal of Advanced Computing Systems · 22.04.2024 · 4 (4) · 97-111

DOI: 10.69987/jacs.2024.40408 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Advanced Computing Systems, online erschienen am 22.04.2024. Verfasst von Ge Liu und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Cloud incidents in AI infrastructure involve overloaded accelerators, brittle service dependencies, distributed databases, logs, and networking paths. Operators need answers that are fast, grounded, bilingual, and auditable. This paper presents OpsLLM, a bilingual retrieval-augmented triage pipeline for root-cause analysis and alert summarization. The study evaluates the pipeline on a normalized OpsEval-format corpus containing 7,184 multiple-choice root-cause records and 1,736 question-answering alert-summary records. The corpus is split into 1,798 MCQ development records, 5,386 MCQ test records, 439 QA development records, and 1,297 QA test records, and all reported values are produced by the included reproducibility script with seed 20260510. Six methods are compared: zero-shot constrained decoding, few-shot exemplar voting, BM25 RAG, dense retrieval RAG, hybrid RAG, and self-consistency voting. The dense RAG path achieved 96.19% top-1 MCQ accuracy, hybrid RAG achieved 95.79%, and self-consistency achieved 96.10%; zero-shot decoding reached 76.29%. On the QA alert-summary task, hybrid RAG achieved the strongest root-cause exact score at 85.81% and the best ROUGE-L at 79.41%. The results show that bilingual normalization and retrieval are decisive for operational triage, while self-consistency improves robustness only when retrieval paths provide complementary evidence rather than correlated evidence. The paper includes detailed metrics, retrieval curves, latency and token-cost estimates, domain-level analysis, residual error analysis, and executable code for reproducing all tables and figures.

Zitieren

APA

Liu, G., Li, C., & Zhang, E. (2024). OpsLLM for Cloud Incident Triage: Bilingual RAG-Based Root Cause Analysis and Alert Summarization for AI Infrastructure Operations. Journal of Advanced Computing Systems, 4(4), 97-111. https://doi.org/10.69987/jacs.2024.40408

Vancouver

Liu G, Li C, Zhang E. OpsLLM for Cloud Incident Triage: Bilingual RAG-Based Root Cause Analysis and Alert Summarization for AI Infrastructure Operations. Journal of Advanced Computing Systems. 2024;4(4):97-111. doi:10.69987/jacs.2024.40408

BibTeX
@article{Liu2024,
  author = {Liu, Ge and Li, Chenyu and Zhang, Eric},
  title = {OpsLLM for Cloud Incident Triage: Bilingual RAG-Based Root Cause Analysis and Alert Summarization for AI Infrastructure Operations},
  journal = {Journal of Advanced Computing Systems},
  year = {2024},
  volume = {4},
  number = {4},
  pages = {97--111},
  doi = {10.69987/jacs.2024.40408}
}
Kostenlos starten →