A-ProS : Towards Reliable Autonomous Programming through Multi-Model Feedback
ACM Transactions on Software Engineering and Methodology · 30.11.2026 · 35 (11) · 1-40
DOI: 10.1145/3820891 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 04.11.2026. Verfasst von Anika Tabassum und 4 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 83 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: NSF.
Abstract (Originalsprache)
Large Language Models (LLMs) demonstrate strong potential for automated code generation, yet their ability to iteratively refine solutions using execution feedback remains underexplored. Competitive programming offers an ideal testbed for this investigation, as it demands end-to-end algorithmic reasoning, precise implementation under strict computational constraints, and complete functional correctness with rigorous evaluation. In this article, we present A-ProS , an autonomous AI agent that solves competitive programming problems through a hybrid multi-model feedback framework separating solution generation from specialized debugging. A-ProS combines ChatGPT-based generators (GPT-4 and GPT-5) with three debugging critics: Codestral-2508, Llama-3.3-70B, and DeepSeek-R1, under a 2 × 3 factorial design. We evaluate six workflows on 367 problems from ICPC World Finals (2011–2024) and Codeforces (rated 1,200–1,800). The results show that GPT-5 workflows improve from 39 initial accepted solutions to 85–90 after three refinement rounds, while GPT-4 improves from 15 to 31–38. A controlled ablation on 47 problems shows that stateful refinement outperforms stateless approaches by 8.5–10.6 percentage points and reduces repeated failures by up to 3.5×. Compared to baseline agent loops, A-ProS achieves over 2× greater gains, highlighting the importance of persistent context and multi-model feedback for reliable autonomous program synthesis.
Förderung
- NSF
Zitieren
APA
Tabassum, A., Hossain, M. S., Arefin, M. F., Islam, T., & Zaman, T. S. (2026). A-ProS : Towards Reliable Autonomous Programming through Multi-Model Feedback. ACM Transactions on Software Engineering and Methodology, 35(11), 1-40. https://doi.org/10.1145/3820891
Vancouver
Tabassum A, Hossain MS, Arefin MF, Islam T, Zaman TS. A-ProS : Towards Reliable Autonomous Programming through Multi-Model Feedback. ACM Transactions on Software Engineering and Methodology. 2026;35(11):1-40. doi:10.1145/3820891
BibTeX
@article{Tabassum2026,
author = {Tabassum, Anika and Hossain, Md Sifat and Arefin, Md. Fahim and Islam, Tariqul and Zaman, Tarannum Shaila},
title = {A-ProS : Towards Reliable Autonomous Programming through Multi-Model Feedback},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2026},
volume = {35},
number = {11},
pages = {1--40},
doi = {10.1145/3820891}
}
EUVIMED