EUVIMEDEUVIMED entdecken
Crossref

Soft-Label Governance for Distributional Safety in Multi-Agent Systems

Aizierjiang Aiersilan, Raeli Savitt

Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 217-225

DOI: 10.1609/aaaiss.v10i1.43409 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Aizierjiang Aiersilan und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-agent AI systems exhibit emergent risks that no single agent produces in isolation. Existing safety frameworks rely on binary classifications of agent behavior, which discard the uncertainty inherent in proxy-based evaluation. We introduce SWARM (System-Wide Assessment of Risk in Multi-agent systems), a simulation framework that replaces binary good/bad labels with soft probabilistic labels p = P(v = +1) in [0, 1], enabling continuous-valued payoffs, toxicity metrics, and governance interventions. SWARM implements a modular governance engine with configurable levers (transaction taxes, circuit breakers, reputation decay, and random audits) and quantifies their effects using probabilistic metrics, including expected toxicity E[1 - p | accepted] and the quality gap E[p | accepted] - E[p | rejected]. In seven scenarios, each replicated with five seeds, a strict threshold-based governance regime records 40.2% less welfare than the ungoverned baseline at indistinguishable toxicity, and the pattern recurs in a population-controlled lever sweep: removing a 10% transaction tax alone recovers 20.3% of welfare without raising toxicity. Full externality internalization collapses welfare from +262 to -67 while toxicity remains at 0.315, and a circuit-breaker sweep identifies an interior optimum whose toxicity gain costs roughly three percent of welfare per percent removed; no setting in that sweep dominates the ungoverned baseline. Companion experiments show that soft metrics detect proxy gaming by a self-optimizing agent that passes every binary check and that the governance layer applies without modification to live LLM-backed agents (Concordia entities, Claude, GPT-4o Mini). These results indicate that distributional safety requires continuous risk metrics and that governance calibration involves quantifiable tradeoffs between safety and system welfare.

Zitieren

APA

Aiersilan, A., & Savitt, R. (2026). Soft-Label Governance for Distributional Safety in Multi-Agent Systems. Proceedings of the AAAI Symposium Series, 10(1), 217-225. https://doi.org/10.1609/aaaiss.v10i1.43409

Vancouver

Aiersilan A, Savitt R. Soft-Label Governance for Distributional Safety in Multi-Agent Systems. Proceedings of the AAAI Symposium Series. 2026;10(1):217-225. doi:10.1609/aaaiss.v10i1.43409

BibTeX
@article{Aiersilan2026,
  author = {Aiersilan, Aizierjiang and Savitt, Raeli},
  title = {Soft-Label Governance for Distributional Safety in Multi-Agent Systems},
  journal = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {217--225},
  doi = {10.1609/aaaiss.v10i1.43409}
}
Kostenlos starten →