EUVIMEDEUVIMED entdecken
Crossref

Characterizing Deep Learning Package Supply Chains in PyPI: Domains, Clusters, and Disengagement

Kai Gao, Runzhi He, Bing Xie, Minghui Zhou

ACM Transactions on Software Engineering and Methodology · 31.05.2024 · 33 (4) · 1-27

DOI: 10.1145/3640336 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 18.04.2024. Verfasst von Kai Gao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 83 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep learning (DL) frameworks have become the cornerstone of the rapidly developing DL field. Through installation dependencies specified in the distribution metadata, numerous packages directly or transitively depend on DL frameworks, layer after layer, forming DL package supply chains (SCs), which are critical for DL frameworks to remain competitive. However, vital knowledge on how to nurture and sustain DL package SCs is still lacking. Achieving this knowledge may help DL frameworks formulate effective measures to strengthen their SCs to remain competitive and shed light on dependency issues and practices in the DL SC for researchers and practitioners. In this paper, we explore the domains, clusters, and disengagement of packages in two representative PyPI DL package SCs to bridge this knowledge gap. We analyze the metadata of nearly six million PyPI package distributions and construct version-sensitive SCs for two popular DL frameworks: TensorFlow and PyTorch. We find that popular packages (measured by the number of monthly downloads) in the two SCs cover 34 domains belonging to eight categories. Applications , Infrastructure , and Sciences categories account for over 85% of popular packages in either SC and TensorFlow and PyTorch SC have developed specializations on Infrastructure and Applications packages, respectively. We employ the Leiden community detection algorithm and detect 131 and 100 clusters in the two SCs. The clusters mainly exhibit four shapes: Arrow, Star, Tree, and Forest with increasing dependency complexity. Most clusters are Arrow or Star, while Tree and Forest clusters account for most packages (Tensorflow SC: 70.7%, PyTorch SC: 92.9%). We identify three groups of reasons why packages disengage from the SC (i.e., remove the DL framework and its dependents from their installation dependencies): dependency issues, functional improvements, and ease of installation. The most common reason in TensorFlow SC is dependency incompatibility and in PyTorch SC is to simplify functionalities and reduce installation size. Our study provides rich implications for DL framework vendors, researchers, and practitioners on the maintenance and dependency management practices of PyPI DL SCs.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Gao, K., He, R., Xie, B., & Zhou, M. (2024). Characterizing Deep Learning Package Supply Chains in PyPI: Domains, Clusters, and Disengagement. ACM Transactions on Software Engineering and Methodology, 33(4), 1-27. https://doi.org/10.1145/3640336

Vancouver

Gao K, He R, Xie B, Zhou M. Characterizing Deep Learning Package Supply Chains in PyPI: Domains, Clusters, and Disengagement. ACM Transactions on Software Engineering and Methodology. 2024;33(4):1-27. doi:10.1145/3640336

BibTeX
@article{Gao2024,
  author = {Gao, Kai and He, Runzhi and Xie, Bing and Zhou, Minghui},
  title = {Characterizing Deep Learning Package Supply Chains in PyPI: Domains, Clusters, and Disengagement},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2024},
  volume = {33},
  number = {4},
  pages = {1--27},
  doi = {10.1145/3640336}
}
Kostenlos starten →