Toward a Principled Framework for Agent Safety Measurement
ACM SIGOPS Operating Systems Review · 13.07.2026 · 60 (1) · 1-8
DOI: 10.1145/3830422.3830423 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM SIGOPS Operating Systems Review, online erschienen am 13.07.2026. Verfasst von Shuyi Lin und 3 weiteren Autor:innen. Die Arbeit nennt 22 Literaturangaben.
Abstract (Originalsprache)
LLM agents emit actions, not just text, and once taken, those actions often cannot be undone. Yet today's agent-safety evaluations run greedy or a few sampled rollouts and report a single safe/unsafe rate—blind to the long-tail trajectories where unsafe behavior may arise from low-probability but non-negligible actions. We argue agent safety should be measured by search, not sampling. We apply BOA a framework that, given a deployment configuration (model, decoder, prompt, environment, judger, likelihood budget), searches the in-budget trajectory space and reports a safety score: the probability the agent stays safe under the configuration. BOA searches both within a single LLM round and across the agent–environment interaction tree under a given likelihood budget, and makes search practical via batched decoding/judging, prefix caching, and chunked tree expansion. On agent-safety workloads, BOA discovers unsafe trajectories that greedy and sampled evaluations miss. BOA can additionally be used for ranking models, defenses, and attacks, all on the same scale, with manageable GPU costs.
Zitieren
APA
Lin, S., Suri, A., Oprea, A., & Tan, C. (2026). Toward a Principled Framework for Agent Safety Measurement. ACM SIGOPS Operating Systems Review, 60(1), 1-8. https://doi.org/10.1145/3830422.3830423
Vancouver
Lin S, Suri A, Oprea A, Tan C. Toward a Principled Framework for Agent Safety Measurement. ACM SIGOPS Operating Systems Review. 2026;60(1):1-8. doi:10.1145/3830422.3830423
BibTeX
@article{Lin2026,
author = {Lin, Shuyi and Suri, Anshuman and Oprea, Alina and Tan, Cheng},
title = {Toward a Principled Framework for Agent Safety Measurement},
journal = {ACM SIGOPS Operating Systems Review},
year = {2026},
volume = {60},
number = {1},
pages = {1--8},
doi = {10.1145/3830422.3830423}
}
EUVIMED