EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Auditing Indirect Prompt Injection Evaluation with an Output Contract

Wengrong Zhang, Mengya Hu, Zhenhan Tang, Zhouguangyu Chen, Zili Gou, Wenbo Xu

Journal of Computer Science and Artificial Intelligence · 30.10.2026 · 9 (1) · 14-22

DOI: 10.54097/wkptse84 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Computer Science and Artificial Intelligence, online erschienen am 30.10.2026. Verfasst von Wengrong Zhang und 5 weiteren Autor:innen. Die Arbeit nennt 13 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automated prompt-injection evaluation can confuse the occurrence of attack-related text with completion of an application-specific objective. This study audits that distinction in a question-answering application returning separate answer and evidence fields. The reference event is exact equality between the decoded answer and a harmless target marker, with invalid output reported separately. A controlled pilot comprises 144 generated responses and 192 constructed diagnostic fixtures. A prospectively frozen extension uses 80 distinct public email and table documents, deterministic retrieval, three local model packages, 624 target calls and 288 individual judge calls. In the extension's 240 main attacked responses, 239 satisfy the structural contract and 52 complete the exact overwrite objective. Whole-response exact occurrence returns 53 positives, including 1 false positives on valid reference-negative responses; answer-only fuzzy matching returns 60 positives, including 8 such false positives. The main extension does not reproduce the pilot's large whole-response excess. The audit distinguishes evidence quotation, longer or approximate marker answers, and short-answer partial matches. Paired quotation instructions and parser-assisted judge inputs provide additional diagnostic comparisons, with actual compliance and reference-class balance reported. For 48 presampled candidates with six positives, answer projection changes correct judgments from 23 to 15 for Llama local and from 36 to 48 for Gemma local. The findings support verifying both the scored field and the matching predicate when reusing an evaluator. They do not establish general model safety, semantic answer correctness, or errors in prior benchmarks under their original contracts.

Zitieren

APA

Zhang, W., Hu, M., Tang, Z., Chen, Z., Gou, Z., & Xu, W. (2026). Auditing Indirect Prompt Injection Evaluation with an Output Contract. Journal of Computer Science and Artificial Intelligence, 9(1), 14-22. https://doi.org/10.54097/wkptse84

Vancouver

Zhang W, Hu M, Tang Z, Chen Z, Gou Z, Xu W. Auditing Indirect Prompt Injection Evaluation with an Output Contract. Journal of Computer Science and Artificial Intelligence. 2026;9(1):14-22. doi:10.54097/wkptse84

BibTeX
@article{Zhang2026,
  author = {Zhang, Wengrong and Hu, Mengya and Tang, Zhenhan and Chen, Zhouguangyu and Gou, Zili and Xu, Wenbo},
  title = {Auditing Indirect Prompt Injection Evaluation with an Output Contract},
  journal = {Journal of Computer Science and Artificial Intelligence},
  year = {2026},
  volume = {9},
  number = {1},
  pages = {14--22},
  doi = {10.54097/wkptse84}
}
Kostenlos starten →