CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench
Applied Sciences · 30.10.2026 · 16 (21) · 10747
DOI: 10.3390/app162110747 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 30.10.2026. Verfasst von You Zhai und 3 weiteren Autor:innen. Die Arbeit nennt 11 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
CVE-Bench evaluates large language model (LLM) agents on 40 real-world, critical-severity web application CVEs in containerized sandboxes. The strongest reported baseline solves 10.0% of zero-day tasks and 12.5% of one-day tasks after five attempts. Common failures include missed endpoints, incorrect tool configuration, and loss of evidence needed for subsequent actions. We present CVE-Pilot, a code LLM agent that uses a typed state graph to record endpoints, credentials, tool outputs, and failed hypotheses. Coverage-guided planning directs exploration, tool contracts check execution and parse results, and target-aware verification tests the eight standard CVE-Bench attack outcomes. Successful and near-miss sandbox traces support self-training through CVE-TraceBank. On the evaluated tasks, CVE-Pilot achieves Success@5 of 25.0% in the zero-day setting and 32.5% in the one-day setting. CVE-level leave-one-out self-training increases these rates to 32.5% and 40.0%, respectively. The gains are accompanied by higher endpoint coverage, better evidence retention, and fewer exploration failures. The 40-CVE sample yields wide confidence intervals, and vulnerability reasoning remains a major source of failure.
Zitieren
APA
Zhai, Y., Song, S., Wang, P., & Zhang, L. (2026). CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench. Applied Sciences, 16(21), 10747. https://doi.org/10.3390/app162110747
Vancouver
Zhai Y, Song S, Wang P, Zhang L. CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench. Applied Sciences. 2026;16(21):10747. doi:10.3390/app162110747
BibTeX
@article{Zhai2026,
author = {Zhai, You and Song, Shouyou and Wang, Pu and Zhang, Lei},
title = {CVE-Pilot: Failure-Aware Tool Orchestration for Code LLM Agents on CVE-Bench},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {21},
pages = {10747},
doi = {10.3390/app162110747}
}
EUVIMED