From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets
ACM Computing Surveys · 31.07.2026 · 58 (10) · 1-39
DOI: 10.1145/3797033 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 09.03.2026. Verfasst von Hao-Nan Zhu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 243 Literaturangaben.
Abstract (Originalsprache)
Software defect datasets, which are collections of software bugs, are essential resources to facilitate empirical research and enable standardized benchmarking for a wide range of software engineering techniques, including emerging areas like agentic AI-based software development. Over the years, numerous software defect datasets have been developed, providing rich resources for the community, yet making it increasingly difficult to navigate the landscape. This article provides a comprehensive survey of 151 software defect datasets, covering their scope, construction, availability, usability, and practical uses. We also suggest potential opportunities for future research based on our findings, such as addressing underrepresented kinds of defects. A complete catalog of all surveyed software defect datasets is available at https://defect-datasets.github.io/ .
Zitieren
APA
Zhu, H., Furth, R., Pradel, M., & Rubio-González, C. (2026). From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets. ACM Computing Surveys, 58(10), 1-39. https://doi.org/10.1145/3797033
Vancouver
Zhu H, Furth R, Pradel M, Rubio-González C. From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets. ACM Computing Surveys. 2026;58(10):1-39. doi:10.1145/3797033
BibTeX
@article{Zhu2026,
author = {Zhu, Hao-Nan and Furth, Robert and Pradel, Michael and Rubio-González, Cindy},
title = {From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets},
journal = {ACM Computing Surveys},
year = {2026},
volume = {58},
number = {10},
pages = {1--39},
doi = {10.1145/3797033}
}
EUVIMED