EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Large Scale Empirical Analysis on the Adherence Gap between Standards and Tools in SBOM

Chengjie Wang, Jingzheng Wu, Hao Lyu, Xiang Ling, Tianyue Luo, Yanjun Wu, Chen Zhao

ACM Transactions on Software Engineering and Methodology · 31.10.2026 · 35 (10) · 1-37

DOI: 10.1145/3788692 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 18.09.2026. Verfasst von Chengjie Wang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 101 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key R&D Program of China, YuanTu Large Research Infrastructure.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A Software Bill of Materials (SBOM) is a machine-readable artifact that systematically organizes software information, enhancing supply chain transparency and security. To facilitate the exchange and utilization of SBOMs, organizations such as the Linux Foundation and OWASP have proposed SBOM standards. Following standards, organizations have developed tools for generating and utilizing SBOMs. However, limited research has examined the adherence of these SBOM tools to standard specifications, a gap that could lead to compliance failures and disruptions in SBOM utilization. This article presents the first large-scale, two-stage empirical analysis of the adherence gap, using our automated evaluation framework, SAP . The evaluation, comprising a baseline evaluation and a 1-year longitudinal follow-up, covers 55,444 SBOMs generated by six SBOM tools from 3,287 real-world repositories. Our analysis reveals persistent, fundamental limitations in current SBOM tools: (1) inadequate compliance support with policy requirements; (2) poor tool consistencies, including inter-tool consistency rates as low as 7.84% to 12.77% for package detection across languages, and significant longitudinal inconsistency, where tools show low consistency with their own prior versions; and (3) mediocre to poor accuracy for detailed software information, e.g., accuracy of package licenses below 20%. We analyze the root causes of these gaps and provide practical solutions. All the code, replication Docker image, evaluation results are open sourced at GitHub and Zenodo for further researches (available at https://github.com/dw763j/SAP and https://doi.org/10.5281/zenodo.14998624 ).

Förderung

  • National Key R&D Program of China
  • YuanTu Large Research Infrastructure

Zitieren

APA

Wang, C., Wu, J., Lyu, H., Ling, X., Luo, T., Wu, Y., & Zhao, C. (2026). A Large Scale Empirical Analysis on the Adherence Gap between Standards and Tools in SBOM. ACM Transactions on Software Engineering and Methodology, 35(10), 1-37. https://doi.org/10.1145/3788692

Vancouver

Wang C, Wu J, Lyu H, Ling X, Luo T, Wu Y, et al. A Large Scale Empirical Analysis on the Adherence Gap between Standards and Tools in SBOM. ACM Transactions on Software Engineering and Methodology. 2026;35(10):1-37. doi:10.1145/3788692

BibTeX
@article{Wang2026,
  author = {Wang, Chengjie and Wu, Jingzheng and Lyu, Hao and Ling, Xiang and Luo, Tianyue and Wu, Yanjun and Zhao, Chen},
  title = {A Large Scale Empirical Analysis on the Adherence Gap between Standards and Tools in SBOM},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2026},
  volume = {35},
  number = {10},
  pages = {1--37},
  doi = {10.1145/3788692}
}
Kostenlos starten →