EUVIMEDEUVIMED entdecken
Crossref

Toward a Principled Framework for Agent Safety Measurement

Shuyi Lin, Anshuman Suri, Alina Oprea, Cheng Tan

ACM SIGOPS Operating Systems Review · 13.07.2026 · 60 (1) · 1-8

DOI: 10.1145/3830422.3830423 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM SIGOPS Operating Systems Review, online erschienen am 13.07.2026. Verfasst von Shuyi Lin und 3 weiteren Autor:innen. Die Arbeit nennt 22 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

LLM agents emit actions, not just text, and once taken, those actions often cannot be undone. Yet today's agent-safety evaluations run greedy or a few sampled rollouts and report a single safe/unsafe rate—blind to the long-tail trajectories where unsafe behavior may arise from low-probability but non-negligible actions. We argue agent safety should be measured by search, not sampling. We apply BOA a framework that, given a deployment configuration (model, decoder, prompt, environment, judger, likelihood budget), searches the in-budget trajectory space and reports a safety score: the probability the agent stays safe under the configuration. BOA searches both within a single LLM round and across the agent–environment interaction tree under a given likelihood budget, and makes search practical via batched decoding/judging, prefix caching, and chunked tree expansion. On agent-safety workloads, BOA discovers unsafe trajectories that greedy and sampled evaluations miss. BOA can additionally be used for ranking models, defenses, and attacks, all on the same scale, with manageable GPU costs.

Zitieren

APA

Lin, S., Suri, A., Oprea, A., & Tan, C. (2026). Toward a Principled Framework for Agent Safety Measurement. ACM SIGOPS Operating Systems Review, 60(1), 1-8. https://doi.org/10.1145/3830422.3830423

Vancouver

Lin S, Suri A, Oprea A, Tan C. Toward a Principled Framework for Agent Safety Measurement. ACM SIGOPS Operating Systems Review. 2026;60(1):1-8. doi:10.1145/3830422.3830423

BibTeX
@article{Lin2026,
  author = {Lin, Shuyi and Suri, Anshuman and Oprea, Alina and Tan, Cheng},
  title = {Toward a Principled Framework for Agent Safety Measurement},
  journal = {ACM SIGOPS Operating Systems Review},
  year = {2026},
  volume = {60},
  number = {1},
  pages = {1--8},
  doi = {10.1145/3830422.3830423}
}
Kostenlos starten →