Conformal Constrained Policy Optimization for Cost-Effective LLM Agents
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25446-25453
DOI: 10.1609/aaai.v40i30.39739 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Wenwen Si und 3 weiteren Autor:innen.
Abstract (Originalsprache)
While large language models (LLMs) have recently made tremendous progress towards solving challenging AI problems, they have done so at increasingly steep computational and API costs. We propose a novel strategy where we combine multiple LLM models with varying cost/accuracy tradeoffs in an agentic manner, where models and tools are run in sequence as determined by an orchestration model to minimize cost subject to a user-specified level of reliability; this constraint is formalized using conformal prediction to provide guarantees. To solve this problem, we propose Conformal Constrained Policy Optimization (CCPO), a training paradigm that integrates constrained policy optimization with off-policy reinforcement learning and recent advances in online conformal prediction. CCPO jointly optimizes a cost-aware policy (score function) and an adaptive threshold. Across two multi-hop question answering benchmarks, CCPO achieves up to a 30% cost reduction compared to other cost-aware baselines and LLM-guided methods without compromising reliability. Our approach provides a principled and practical framework for deploying LLM agents that are significantly more cost-effective while maintaining reliability.
Zitieren
APA
Si, W., Jang, S., Lee, I., & Bastani, O. (2026). Conformal Constrained Policy Optimization for Cost-Effective LLM Agents. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25446-25453. https://doi.org/10.1609/aaai.v40i30.39739
Vancouver
Si W, Jang S, Lee I, Bastani O. Conformal Constrained Policy Optimization for Cost-Effective LLM Agents. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25446-25453. doi:10.1609/aaai.v40i30.39739
BibTeX
@article{Si2026,
author = {Si, Wenwen and Jang, Sooyong and Lee, Insup and Bastani, Osbert},
title = {Conformal Constrained Policy Optimization for Cost-Effective LLM Agents},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {30},
pages = {25446--25453},
doi = {10.1609/aaai.v40i30.39739}
}
EUVIMED