EUVIMEDEUVIMED entdecken
Crossref

Conformal Constrained Policy Optimization for Cost-Effective LLM Agents

Wenwen Si, Sooyong Jang, Insup Lee, Osbert Bastani

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25446-25453

DOI: 10.1609/aaai.v40i30.39739 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Wenwen Si und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

While large language models (LLMs) have recently made tremendous progress towards solving challenging AI problems, they have done so at increasingly steep computational and API costs. We propose a novel strategy where we combine multiple LLM models with varying cost/accuracy tradeoffs in an agentic manner, where models and tools are run in sequence as determined by an orchestration model to minimize cost subject to a user-specified level of reliability; this constraint is formalized using conformal prediction to provide guarantees. To solve this problem, we propose Conformal Constrained Policy Optimization (CCPO), a training paradigm that integrates constrained policy optimization with off-policy reinforcement learning and recent advances in online conformal prediction. CCPO jointly optimizes a cost-aware policy (score function) and an adaptive threshold. Across two multi-hop question answering benchmarks, CCPO achieves up to a 30% cost reduction compared to other cost-aware baselines and LLM-guided methods without compromising reliability. Our approach provides a principled and practical framework for deploying LLM agents that are significantly more cost-effective while maintaining reliability.

Zitieren

APA

Si, W., Jang, S., Lee, I., & Bastani, O. (2026). Conformal Constrained Policy Optimization for Cost-Effective LLM Agents. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25446-25453. https://doi.org/10.1609/aaai.v40i30.39739

Vancouver

Si W, Jang S, Lee I, Bastani O. Conformal Constrained Policy Optimization for Cost-Effective LLM Agents. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25446-25453. doi:10.1609/aaai.v40i30.39739

BibTeX
@article{Si2026,
  author = {Si, Wenwen and Jang, Sooyong and Lee, Insup and Bastani, Osbert},
  title = {Conformal Constrained Policy Optimization for Cost-Effective LLM Agents},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {30},
  pages = {25446--25453},
  doi = {10.1609/aaai.v40i30.39739}
}
Kostenlos starten →