Revisiting Vulnerability Patch Identification on Data in the Wild
ACM Transactions on Software Engineering and Methodology · 07.04.2026
DOI: 10.1145/3806044 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 07.04.2026. Verfasst von Ivana Clairine Irsan und 9 weiteren Autor:innen. Die Arbeit nennt 79 Literaturangaben.
Abstract (Originalsprache)
Attacks can exploit zero-day or one-day vulnerabilities that are not publicly disclosed. To detect these vulnerabilities, security researchers monitor development activities in open-source repositories to identify unreported security patches. The sheer volume of commits makes this task infeasible to accomplish manually. Consequently, security patch detectors commonly trained and evaluated on security patches linked from vulnerability reports in the National Vulnerability Database (NVD). In this study, we assess the effectiveness of these detectors when applied in-the-wild. Our results show that models trained on NVD-derived data show substantially decreased performance, with decreases in F1-score of up to 90% when tested on in-the-wild security patches, rendering them impractical for real-world use. An analysis comparing security patches identified in-the-wild and commits linked from NVD reveals that they can be easily distinguished from each other. Security patches associated with NVD have different distribution of commit messages, vulnerability types, and composition of changes. These differences suggest that NVD may be unsuitable as the sole source of data for training models to detect security patches. We find that constructing a dataset that combines security patches from NVD data with a small subset of manually identified security patches can improve model robustness.
Zitieren
APA
Irsan, I. C., Widyasari, R., Zhang, T., Huang, H., Thung, F., Li, Y., Shar, L. K., Ouh, E. L., Kang, H. J., & Lo, D. (2026). Revisiting Vulnerability Patch Identification on Data in the Wild. ACM Transactions on Software Engineering and Methodology. https://doi.org/10.1145/3806044
Vancouver
Irsan IC, Widyasari R, Zhang T, Huang H, Thung F, Li Y, et al. Revisiting Vulnerability Patch Identification on Data in the Wild. ACM Transactions on Software Engineering and Methodology. 2026. doi:10.1145/3806044
BibTeX
@article{Irsan2026,
author = {Irsan, Ivana Clairine and Widyasari, Ratnadira and Zhang, Ting and Huang, Huihui and Thung, Ferdian and Li, Yikun and Shar, Lwin Khin and Ouh, Eng Lieh and Kang, Hong Jin and Lo, David},
title = {Revisiting Vulnerability Patch Identification on Data in the Wild},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2026},
doi = {10.1145/3806044}
}
EUVIMED