Contextualized Code Pretraining for Code Generation
ACM Transactions on Software Engineering and Methodology · 03.11.2026
DOI: 10.1145/3860400 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 03.11.2026. Verfasst von Chen Liu und 5 weiteren Autor:innen. Die Arbeit nennt 65 Literaturangaben.
Abstract (Originalsprache)
As code generation becomes increasingly central to improving software development efficiency, modern code models are largely trained and evaluated on code with natural-language descriptions. In real projects, developers often implement missing functions under limited project-specific artifacts, while the local call-site context is already available in the surrounding code. This usage context provides actionable cues about expected behavior, but existing models are not explicitly optimized to leverage it reliably, leading to implementations that may not integrate smoothly with surrounding usage in repository settings. In this work, we propose contextualized code pretraining, an invocation-aware framework that integrates calling context into both the training and evaluation of code models. Using static analysis, we automatically extract large-scale caller–callee pairs from real repositories to construct pretraining tasks and benchmarks that condition generation on the calling context. We train CallerGen , the first code models pretrained with invocation-aware objectives spanning multiple sizes, and evaluate them on CallerEval , a new benchmark featuring realistic scenarios. Experiments across our two primary benchmarks show that CallerGen outperforms comparable-scale models and remains competitive with larger ones. Our 220M and 0.5B models achieve 16.58% and 22.81% pass@1, surpassing baselines on CallerEval . These results highlight the importance of calling context in realistic code generation.
Zitieren
APA
Liu, C., Liang, Q., Zhang, H., Sun, Z., Zhang, Y., & Zhang, L. (2026). Contextualized Code Pretraining for Code Generation. ACM Transactions on Software Engineering and Methodology. https://doi.org/10.1145/3860400
Vancouver
Liu C, Liang Q, Zhang H, Sun Z, Zhang Y, Zhang L. Contextualized Code Pretraining for Code Generation. ACM Transactions on Software Engineering and Methodology. 2026. doi:10.1145/3860400
BibTeX
@article{Liu2026,
author = {Liu, Chen and Liang, Qingyuan and Zhang, Hanwen and Sun, Zeyu and Zhang, Yakun and Zhang, Lu},
title = {Contextualized Code Pretraining for Code Generation},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2026},
doi = {10.1145/3860400}
}
EUVIMED