EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-Coded Credentials

Yizhan Huang, Yichen Li, Weibin Wu, Jianping Zhang, Michael R. Lyu

Proceedings of the ACM on Software Engineering · 12.07.2024 · 1 (FSE) · 2515-2537

DOI: 10.1145/3660818 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 12.07.2024. Verfasst von Yizhan Huang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The National Natural Science Foundation of China, The Research Grants Council of the Hong Kong Special Administrative Region, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Neural Code Completion Tools (NCCTs) have reshaped the field of software engineering, which are built upon the language modeling technique and can accurately suggest contextually relevant code snippets. However, language models may emit the training data verbatim during inference with appropriate prompts. This memorization property raises privacy concerns of NCCTs about hard-coded credential leakage, leading to unauthorized access to applications, systems, or networks. Therefore, to answer whether NCCTs will emit the hard-coded credential, we propose an evaluation tool called H ard-coded C redential R evealer (HCR). HCR constructs test prompts based on GitHub code files with credentials to reveal the memorization phenomenon of NCCTs. Then, HCR designs four filters to filter out ill-formatted credentials. Finally, HCR directly checks the validity of a set of non-sensitive credentials. We apply HCR to evaluate three representative types of NCCTs: Commercial NCCTs, open-source models, and chatbots with code completion capability. Our experimental results show that NCCTs can not only return the precise piece of their training data but also inadvertently leak additional secret strings. Notably, two valid credentials were identified during our experiments. Therefore, HCR raises a severe privacy concern about the potential leakage of hard-coded credentials in the training data of commercial NCCTs. All artifacts and data are released for future research purposes in https://github.com/HCR-Repo/HCR .

Förderung

  • The National Natural Science Foundation of China
  • The Research Grants Council of the Hong Kong Special Administrative Region, China

Zitieren

APA

Huang, Y., Li, Y., Wu, W., Zhang, J., & Lyu, M. R. (2024). Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-Coded Credentials. Proceedings of the ACM on Software Engineering, 1(FSE), 2515-2537. https://doi.org/10.1145/3660818

Vancouver

Huang Y, Li Y, Wu W, Zhang J, Lyu MR. Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-Coded Credentials. Proceedings of the ACM on Software Engineering. 2024;1(FSE):2515-2537. doi:10.1145/3660818

BibTeX
@article{Huang2024,
  author = {Huang, Yizhan and Li, Yichen and Wu, Weibin and Zhang, Jianping and Lyu, Michael R.},
  title = {Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-Coded Credentials},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2024},
  volume = {1},
  number = {FSE},
  pages = {2515--2537},
  doi = {10.1145/3660818}
}
Kostenlos starten →