Many hands make light work: An LLM-based multi-agent system for detecting malicious PyPI packages
Journal of Systems and Software · 2026-06 · 236 · 112792
DOI: 10.1016/j.jss.2026.112792 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Systems and Software, 2026-06. Verfasst von Muhammad Umar Zeshan und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Governo Italiano Ministero dell'Istruzione dell'Università e della Ricerca, Horizon Europe, Ministero dell’Istruzione, dell’Università e della Ricerca.
Abstract (Originalsprache)
Malicious code in open-source repositories such as PyPI poses a growing threat to software supply chains. Traditional rule-based tools often overlook the semantic patterns in source code that are crucial for identifying adversarial components. Large language models (LLMs) show promise for software analysis, yet their use in interpretable and modular security pipelines remains limited. This paper presents LAMPS, a multi-agent system that employs collaborative LLMs to detect malicious PyPI packages. The system consists of four role-specific agents for package retrieval, file extraction, classification, and verdict aggregation, coordinated through the CrewAI framework. A prototype combines a fine-tuned CodeBERT model for classification with LLaMA-3 agents for contextual reasoning. LAMPS has been evaluated on two complementary datasets: D1, a balanced collection of 6,000 setup.py files, and D2, a realistic multi-file dataset with 1,296 files and natural class imbalance. On D1, LAMPS achieves 97.7% accuracy, surpassing MPHunter--one of the state-of-the-art approaches. On D2, it reaches 99.5% accuracy and 99.5% balanced accuracy, outperforming RAG-based approaches and fine-tuned single-agent baselines. McNemar's test confirmed these improvements as highly significant. The results demonstrate the feasibility of distributed LLM reasoning for malicious code detection and highlight the benefits of modular multi-agent designs in software supply chain security.
Förderung
- Governo Italiano Ministero dell'Istruzione dell'Università e della Ricerca
- Horizon Europe
- Ministero dell’Istruzione, dell’Università e della Ricerca
- Ministero dell'Università e della Ricerca
- European Commission
Zitieren
APA
Zeshan, M. U., Ibiyo, M., Sipio, C. D., Nguyen, P. T., & Ruscio, D. D. (2026). Many hands make light work: An LLM-based multi-agent system for detecting malicious PyPI packages. Journal of Systems and Software, 236, 112792. https://doi.org/10.1016/j.jss.2026.112792
Vancouver
Zeshan MU, Ibiyo M, Sipio CD, Nguyen PT, Ruscio DD. Many hands make light work: An LLM-based multi-agent system for detecting malicious PyPI packages. Journal of Systems and Software. 2026;236:112792. doi:10.1016/j.jss.2026.112792
BibTeX
@article{Zeshan2026,
author = {Zeshan, Muhammad Umar and Ibiyo, Motunrayo and Sipio, Claudio Di and Nguyen, Phuong T. and Ruscio, Davide Di},
title = {Many hands make light work: An LLM-based multi-agent system for detecting malicious PyPI packages},
journal = {Journal of Systems and Software},
year = {2026},
volume = {236},
pages = {112792},
doi = {10.1016/j.jss.2026.112792}
}
EUVIMED