EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection

Lei Yu, Zhirong Huang, Hang Yuan, Shiqi Cheng, Li Yang, Fengjun Zhang, Chenjie Shen, Jiajia Ma, Jingyuan Zhang, Junyi Lu, Chun Zuo

Proceedings of the ACM on Software Engineering · 22.06.2025 · 2 (ISSTA) · 182-205

DOI: 10.1145/3728878 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 22.06.2025. Verfasst von Lei Yu und 10 weiteren Autor:innen. Die Arbeit wurde bisher 44-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, Alliance of International Science Organizations.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Smart contract vulnerability detection is a critical challenge in the rapidly evolving blockchain landscape. Existing vulnerability detection methods face two main issues: (1) Existing datasets lack comprehensiveness and sufficient quality, with limited vulnerability type coverage and insufficient distinction between high-quality and low-quality explanations for preference learning. (2) Large language models (LLMs) oftenstruggle with accurately interpreting specific concepts in smart contract security. Through our empirical analysis, we found that even after continual pre-training and supervised fine-tuning, LLMs still exhibit limitations in precisely understanding the execution order of state changes in smart contracts, which can lead to incorrect vulnerability explanations despite making correct detection decisions. These limitations result in poor detection performance, leading to potentially severe financial losses. To address these challenges, we propose Smart-LLaMA-DPO, an advanced detection method based on the LLaMA-3.1-8B. First, we construct a comprehensive dataset covering four vulnerability types and machine-unauditable vulnerabilities, containing labels, detailed explanations, and precise vulnerability locations for Supervised Fine-Tuning (SFT), as well as paired high-quality and low-quality outputs for Direct Preference Optimization (DPO). Second, we perform continual pre-training using large-scale smart contract code to enhance the LLM's understanding of specific security practices in smart contracts. Futhermore, we conduct supervised fine-tuning with our comprehensive dataset. Finally, we apply DPO, which leverages human feedback to improve the quality of generated explanations. Smart-LLaMA-DPO utilizes a specially designed loss function that encourages the LLM to increase the probability of preferred outputs while decreasing the probability of non-preferred outputs, thereby enhancing the LLM's ability to generate highquality explanations. We evaluate Smart-LLaMA-DPO on four major vulnerability types: reentrancy, timestamp dependence, integer overflow/underflow, and delegatecall, as well as machine-unauditable vulnerabilities. Our method significantly outperforms state-of-the-art baselines, with average improvements of 10.43% in F1 score and 7.87% in accuracy. Moreover, both LLM evaluation and human evaluation demonstrate the superior quality of explanations generated by Smart-LLaMA-DPO in terms of correctness, thoroughness, and clarity.

Förderung

  • National Key Research and Development Program of China
  • Alliance of International Science Organizations

Zitieren

APA

Yu, L., Huang, Z., Yuan, H., Cheng, S., Yang, L., Zhang, F., Shen, C., Ma, J., Zhang, J., Lu, J., & Zuo, C. (2025). Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection. Proceedings of the ACM on Software Engineering, 2(ISSTA), 182-205. https://doi.org/10.1145/3728878

Vancouver

Yu L, Huang Z, Yuan H, Cheng S, Yang L, Zhang F, et al. Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection. Proceedings of the ACM on Software Engineering. 2025;2(ISSTA):182-205. doi:10.1145/3728878

BibTeX
@article{Yu2025,
  author = {Yu, Lei and Huang, Zhirong and Yuan, Hang and Cheng, Shiqi and Yang, Li and Zhang, Fengjun and Shen, Chenjie and Ma, Jiajia and Zhang, Jingyuan and Lu, Junyi and Zuo, Chun},
  title = {Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2025},
  volume = {2},
  number = {ISSTA},
  pages = {182--205},
  doi = {10.1145/3728878}
}
Kostenlos starten →