EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench

You Zhai, Shouyou Song, Pu Wang, Lei Zhang

Applied Sciences · 30.10.2026 · 16 (21) · 10747

DOI: 10.3390/app162110747 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 30.10.2026. Verfasst von You Zhai und 3 weiteren Autor:innen. Die Arbeit nennt 11 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

CVE-Bench evaluates large language model (LLM) agents on 40 real-world, critical-severity web application CVEs in containerized sandboxes. The strongest reported baseline solves 10.0% of zero-day tasks and 12.5% of one-day tasks after five attempts. Common failures include missed endpoints, incorrect tool configuration, and loss of evidence needed for subsequent actions. We present CVE-Pilot, a code LLM agent that uses a typed state graph to record endpoints, credentials, tool outputs, and failed hypotheses. Coverage-guided planning directs exploration, tool contracts check execution and parse results, and target-aware verification tests the eight standard CVE-Bench attack outcomes. Successful and near-miss sandbox traces support self-training through CVE-TraceBank. On the evaluated tasks, CVE-Pilot achieves Success@5 of 25.0% in the zero-day setting and 32.5% in the one-day setting. CVE-level leave-one-out self-training increases these rates to 32.5% and 40.0%, respectively. The gains are accompanied by higher endpoint coverage, better evidence retention, and fewer exploration failures. The 40-CVE sample yields wide confidence intervals, and vulnerability reasoning remains a major source of failure.

Zitieren

APA

Zhai, Y., Song, S., Wang, P., & Zhang, L. (2026). CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench. Applied Sciences, 16(21), 10747. https://doi.org/10.3390/app162110747

Vancouver

Zhai Y, Song S, Wang P, Zhang L. CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench. Applied Sciences. 2026;16(21):10747. doi:10.3390/app162110747

BibTeX
@article{Zhai2026,
  author = {Zhai, You and Song, Shouyou and Wang, Pu and Zhang, Lei},
  title = {CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {21},
  pages = {10747},
  doi = {10.3390/app162110747}
}
Kostenlos starten →