BackportBench: A Multilingual Benchmark for Automated Patch Backporting
Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 700-722
DOI: 10.1145/3797127 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Zhiqing Zhong und 2 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 79 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Guangdong Basic and Applied Basic Research Foundation.
Abstract (Originalsprache)
Many modern software projects evolve rapidly to incorporate new features and security patches. It is important for users to update their dependencies to safer versions, but many still use older, vulnerable package versions because upgrading can be difficult and may break their existing codebase. Software developers can mitigate this problem by backporting security patches to older releases. However, manually backporting is time-consuming and error-prone. The effectiveness of existing automated backporting techniques on general software remains unclear since they typically target only code-hunk or function-level patch porting scenarios and are evaluated with imperfect metrics. To facilitate the development and evaluation of automated backporting techniques, we introduce BackportBench, the first comprehensive benchmark suite for patch backporting problem. BackportBench is a multilingual benchmark that contains 202 patch backporting problems from PyPI, Maven, and npm, each with executable Docker environments and relevant test cases. We evaluated existing patch porting methods and LLM-based techniques that have the potential to adapt to this task using BackportBench. The results show that the agentic method has outperformed traditional patch porting methods, especially on cases that require logical and structural changes. However, the performance varies across different programming languages. Based on the findings, we draw several implications for researchers and software practitioners in future work on automated backporting.
Förderung
- Guangdong Basic and Applied Basic Research Foundation
Zitieren
APA
Zhong, Z., Huang, J., & He, P. (2026). BackportBench: A Multilingual Benchmark for Automated Patch Backporting. Proceedings of the ACM on Software Engineering, 3(FSE), 700-722. https://doi.org/10.1145/3797127
Vancouver
Zhong Z, Huang J, He P. BackportBench: A Multilingual Benchmark for Automated Patch Backporting. Proceedings of the ACM on Software Engineering. 2026;3(FSE):700-722. doi:10.1145/3797127
BibTeX
@article{Zhong2026,
author = {Zhong, Zhiqing and Huang, Jiaming and He, Pinjia},
title = {BackportBench: A Multilingual Benchmark for Automated Patch Backporting},
journal = {Proceedings of the ACM on Software Engineering},
year = {2026},
volume = {3},
number = {FSE},
pages = {700--722},
doi = {10.1145/3797127}
}
EUVIMED