Autonomous Evaluation Architectures: Multi-Agent LLM Pipelines, Browser-Grounded Testing: Programmatic Alignment via DSPy, and Adversarial Robustness in Production Orchestration Systems
Computer Fraud and Security · 12.05.2026 · 1268-1279
DOI: 10.52710/cfs.1067 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computer Fraud and Security, online erschienen am 12.05.2026. Verfasst von Venkata Chandra Sekhar Sastry Chilkuri.
Abstract (Originalsprache)
Evaluating multi-agent large language model systems requires fundamentally different approaches than evaluating single-model outputs. Conventional benchmarks assess isolated model capabilities in controlled conditions, but production multi-agent pipelines exhibit emergent failure modes that only manifest through agent interactions across pipeline stages. An individual agent may produce valid output that, when consumed by a downstream agent, leads to semantically incorrect or structurally broken final artifacts, a class of failures that per-agent evaluation cannot detect by design. This article introduces AgentForge-Eval, a closed-loop evaluation architecture that combines browser-grounded execution testing, multi-layer deterministic and semantic assertion frameworks, and programmatic prompt alignment to autonomously detect, diagnose, and remediate multi-agent failures. Unlike static benchmarks that assess what models produce, AgentForge-Eval tests what multi-agent outputs actually do by executing generated artifacts in headless browser environments and feeding runtime results back into an iterative fix loop with formal convergence guarantees. Deployment in a production multi-agent pipeline demonstrates substantial improvements in first-pass acceptance rates, significant reductions in iterations required before approval, and detection of a materially larger share of failures than semantic judge evaluation captures alone. Programmatic optimization using the full evaluation stack as its objective achieves additional composite metric gains through automated cross-stage prompt alignment. The framework contributes a formal taxonomy of multi-agent failure modes and empirical evidence that browser-grounded evaluation captures a failure class that proxy-metric assessment cannot reach.
Zitieren
APA
Chilkuri, V. C. S. S. (2026). Autonomous Evaluation Architectures: Multi-Agent LLM Pipelines, Browser-Grounded Testing: Programmatic Alignment via DSPy, and Adversarial Robustness in Production Orchestration Systems. Computer Fraud and Security, 1268-1279. https://doi.org/10.52710/cfs.1067
Vancouver
Chilkuri VCSS. Autonomous Evaluation Architectures: Multi-Agent LLM Pipelines, Browser-Grounded Testing: Programmatic Alignment via DSPy, and Adversarial Robustness in Production Orchestration Systems. Computer Fraud and Security. 2026:1268-1279. doi:10.52710/cfs.1067
BibTeX
@article{Chilkuri2026,
author = {Chilkuri, Venkata Chandra Sekhar Sastry},
title = {Autonomous Evaluation Architectures: Multi-Agent LLM Pipelines, Browser-Grounded Testing: Programmatic Alignment via DSPy, and Adversarial Robustness in Production Orchestration Systems},
journal = {Computer Fraud and Security},
year = {2026},
pages = {1268--1279},
doi = {10.52710/cfs.1067}
}
EUVIMED