EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating Indirect Prompt Injection Defenses in Tool-Using LLM Agents: Security, Utility, and Replication

Adil Khan, Khaled AlKhanbashi, Azza Mohamed

Computers · 31.08.2026 · 15 (9) · 570

DOI: 10.3390/computers15090570 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers, online erschienen am 31.08.2026. Verfasst von Adil Khan und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language model (LLM) agents that retrieve external content and use tools are vulnerable to indirect prompt injection, in which untrusted content contains instructions intended to influence agent behavior. We evaluated four defenses and an undefended control across GPT-5.4, GPT-5.4-mini, and Claude Sonnet 4.6 on the AgentDojo banking benchmark (Tool Filter was evaluated only for the OpenAI models), reporting attack success rate (ASR), benign utility, utility under attack, operational measures, and two independent benchmark replications. Raw undefended ASR was 0/288 for GPT-5.4, 11/288 for GPT-5.4-mini, and 1/288 for Claude Sonnet 4.6; these cross-model differences require cautious interpretation because benchmark goals were not equally reachable across models. For GPT-5.4-mini, the Prompt Injection Detector and Tool Filter were associated with lower observed ASRs but also lower benign utility, and Tool Filter restricted available actions. None of the four paired GPT-5.4-mini comparisons reached significance after Holm correction; only Tool Filter had an unadjusted p-value below 0.05. Benign utility was more stable across runs than individual low-frequency attack outcomes. The findings show that defense evaluation should report attack outcomes, goal feasibility, legitimate-task utility, action availability, operational measures, and run-to-run variation. Results are limited to the evaluated benchmark, models, defenses, and conditions.

Zitieren

APA

Khan, A., AlKhanbashi, K., & Mohamed, A. (2026). Evaluating Indirect Prompt Injection Defenses in Tool-Using LLM Agents: Security, Utility, and Replication. Computers, 15(9), 570. https://doi.org/10.3390/computers15090570

Vancouver

Khan A, AlKhanbashi K, Mohamed A. Evaluating Indirect Prompt Injection Defenses in Tool-Using LLM Agents: Security, Utility, and Replication. Computers. 2026;15(9):570. doi:10.3390/computers15090570

BibTeX
@article{Khan2026,
  author = {Khan, Adil and AlKhanbashi, Khaled and Mohamed, Azza},
  title = {Evaluating Indirect Prompt Injection Defenses in Tool-Using LLM Agents: Security, Utility, and Replication},
  journal = {Computers},
  year = {2026},
  volume = {15},
  number = {9},
  pages = {570},
  doi = {10.3390/computers15090570}
}
Kostenlos starten →