EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Code Change Intention, Development Artifact, and History Vulnerability: Putting Them Together for Vulnerability Fix Detection by LLM

Xu Yang, Wenhan Zhu, Michael Pacheco, Jiayuan Zhou, Shaowei Wang, Xing Hu, Kui Liu

Proceedings of the ACM on Software Engineering · 19.06.2025 · 2 (FSE) · 489-510

DOI: 10.1145/3715738 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 19.06.2025. Verfasst von Xu Yang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 77 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Detecting vulnerability fix commits in open-source software is crucial for maintaining software security. To help OSS identify vulnerability fix commits, several automated approaches are developed. However, existing approaches like VulFixMiner and CoLeFunDa, focus solely on code changes, neglecting essential context from development artifacts. Tools like Vulcurator, which integrates issue reports, fail to leverage semantic associations between different development artifacts (e.g., pull requests and history vulnerability fixes). Moreover, they miss vulnerability fixes in tangled commits and lack explanations, limiting practical use. Hence to address those limitations, we propose LLM4VFD, a novel framework that leverages Large Language Models (LLMs) enhanced with Chain-of-Thought reasoning and In-Context Learning to improve the accuracy of vulnerability fix detection. LLM4VFD comprises three components: (1) Code Change Intention, which analyzes commit summaries, purposes, and implications using Chain-of-Thought reasoning; (2) Development Artifact, which incorporates context from related issue reports and pull requests; (3) Historical Vulnerability, which retrieves similar past vulnerability fixes to enrich context. More importantly, on top of the prediction, LLM4VFD also provides a detailed analysis and explanation to help security experts understand the rationale behind the decision. We evaluated LLM4VFD against state-of-the-art techniques, including Pre-trained Language Model-based approaches and vanilla LLMs, using a newly collected dataset, BigVulFixes. Experimental results demonstrate that LLM4VFD significantly outperforms the best-performed existing approach by 68.1%–145.4%. Furthermore, We conducted a user study with security experts, showing that the analysis generated by LLM4VFD improves the efficiency of vulnerability fix identification.

Zitieren

APA

Yang, X., Zhu, W., Pacheco, M., Zhou, J., Wang, S., Hu, X., & Liu, K. (2025). Code Change Intention, Development Artifact, and History Vulnerability: Putting Them Together for Vulnerability Fix Detection by LLM. Proceedings of the ACM on Software Engineering, 2(FSE), 489-510. https://doi.org/10.1145/3715738

Vancouver

Yang X, Zhu W, Pacheco M, Zhou J, Wang S, Hu X, et al. Code Change Intention, Development Artifact, and History Vulnerability: Putting Them Together for Vulnerability Fix Detection by LLM. Proceedings of the ACM on Software Engineering. 2025;2(FSE):489-510. doi:10.1145/3715738

BibTeX
@article{Yang2025,
  author = {Yang, Xu and Zhu, Wenhan and Pacheco, Michael and Zhou, Jiayuan and Wang, Shaowei and Hu, Xing and Liu, Kui},
  title = {Code Change Intention, Development Artifact, and History Vulnerability: Putting Them Together for Vulnerability Fix Detection by LLM},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2025},
  volume = {2},
  number = {FSE},
  pages = {489--510},
  doi = {10.1145/3715738}
}
Kostenlos starten →