Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study
ACM Transactions on Software Engineering and Methodology · 30.11.2025 · 34 (8) · 1-34
DOI: 10.1145/3716848 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 06.10.2025. Verfasst von Yujia Fu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 80-mal zitiert (Crossref) und nennt 84 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Major Science and Technology Project of Hubei, Supercomputing Center of Wuhan University.
Abstract (Originalsprache)
Modern code generation tools utilizing AI models like Large Language Models have gained increased popularity due to their ability to produce functional code. However, their usage presents security challenges, often resulting in insecure code merging into the code base. Thus, evaluating the quality of generated code, especially its security, is crucial. While prior research explored various aspects of code generation, the focus on security has been limited, mostly examining code produced in controlled environments rather than open source development scenarios. To address this gap, we conducted an empirical study, analyzing code snippets generated by GitHub Copilot and two other AI code generation tools (i.e., CodeWhisperer and Codeium) from GitHub projects. Our analysis identified 733 snippets, revealing a high likelihood of security weaknesses, with 29.5% of Python and 24.2% of JavaScript snippets affected. These issues span 43 Common Weakness Enumeration (CWE) categories, including significant ones like CWE-330: Use of Insufficiently Random Values , CWE-94: Improper Control of Generation of Code , and CWE-79: Cross-site Scripting . Notably, eight of those CWEs are among the 2023 CWE Top-25, highlighting their severity. We further examined using Copilot Chat to fix security issues in Copilot-generated code by providing Copilot Chat with warning messages from the static analysis tools, and up to 55.5% of the security issues can be fixed. We finally provide the suggestions for mitigating security issues in generated code.
Förderung
- National Natural Science Foundation of China
- Major Science and Technology Project of Hubei
- Supercomputing Center of Wuhan University
Zitieren
APA
Fu, Y., Liang, P., Tahir, A., Li, Z., Shahin, M., Yu, J., & Chen, J. (2025). Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study. ACM Transactions on Software Engineering and Methodology, 34(8), 1-34. https://doi.org/10.1145/3716848
Vancouver
Fu Y, Liang P, Tahir A, Li Z, Shahin M, Yu J, et al. Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study. ACM Transactions on Software Engineering and Methodology. 2025;34(8):1-34. doi:10.1145/3716848
BibTeX
@article{Fu2025,
author = {Fu, Yujia and Liang, Peng and Tahir, Amjed and Li, Zengyang and Shahin, Mojtaba and Yu, Jiaxin and Chen, Jinfu},
title = {Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2025},
volume = {34},
number = {8},
pages = {1--34},
doi = {10.1145/3716848}
}
EUVIMED