Challenge on Optimization of Context Collection for Code Completion at ASE 2025
ACM SIGSOFT Software Engineering Notes · 04.06.2026 · 51 (2) · 20-26
DOI: 10.1145/3800646.3800652 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM SIGSOFT Software Engineering Notes, online erschienen am 03.06.2026. Verfasst von Dmitry Ustalov und 6 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben.
Abstract (Originalsprache)
The rapid advancement of workflows and methods for software engineering using AI emphasizes the need for a systematic evaluation and analysis of their ability to leverage information from entire projects, particularly in large code bases. In this challenge on optimization of context collection for code completion, organized by JetBrains in collaboration with Mistral AI as part of the ASE 2025 conference, participants developed efficient mechanisms for collecting context from source code repositories to improve fill-in-the-middle code completions for Python and Kotlin. We constructed a large dataset of real-world code in these two programming languages using permissively licensed open-source projects. The submissions were evaluated based on their ability to maximize completion quality for multiple state-of-the-art neural models using the chrF metric. During the public phase of the competition, nineteen teams submitted solutions to the Python track and eight teams submitted solutions to the Kotlin track. In the private phase, six teams competed, of which five submitted papers to the workshop.
Zitieren
APA
Ustalov, D., Bogomolov, E., Bezzubov, A., Golubev, Y., Glukhov, E., Levtsov, G., & Kovalenko, V. (2026). Challenge on Optimization of Context Collection for Code Completion at ASE 2025. ACM SIGSOFT Software Engineering Notes, 51(2), 20-26. https://doi.org/10.1145/3800646.3800652
Vancouver
Ustalov D, Bogomolov E, Bezzubov A, Golubev Y, Glukhov E, Levtsov G, et al. Challenge on Optimization of Context Collection for Code Completion at ASE 2025. ACM SIGSOFT Software Engineering Notes. 2026;51(2):20-26. doi:10.1145/3800646.3800652
BibTeX
@article{Ustalov2026,
author = {Ustalov, Dmitry and Bogomolov, Egor and Bezzubov, Alexander and Golubev, Yaroslav and Glukhov, Evgeniy and Levtsov, Georgii and Kovalenko, Vladimir},
title = {Challenge on Optimization of Context Collection for Code Completion at ASE 2025},
journal = {ACM SIGSOFT Software Engineering Notes},
year = {2026},
volume = {51},
number = {2},
pages = {20--26},
doi = {10.1145/3800646.3800652}
}
EUVIMED