EUVIMEDEUVIMED entdecken
Crossref

Detection of Malicious Codes Generated by Large Language Models: A Comparison of GPT-3.5, GPT-4o, Gemini, and Claude

Meltem Kurt Pehlivanoğlu, Murat Görkem Çoban

International Journal of Information Security Science · 2025

DOI: 10.55859/ijiss.1634763 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Information Security Science, 2025. Verfasst von Meltem Kurt Pehlivanoğlu und 1 weiteren Autor:innen. Die Arbeit nennt 17 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study presents novel machine learning-based approaches for detecting whether source code generated by Large Language Models (LLMs) contains malicious code. To achieve this, comprehensive datasets comprising malicious and benign code samples were created using the GPT-3.5 (ChatGPT), GPT-4o, Gemini, and Claude language models. The extracted code samples were then processed through CodeBERT, CodeT5, and manual feature extraction techniques before being classified using various machine learning algorithms. Experimental results demonstrate that this approach can effectively detect malicious software in code generated by LLMs. This study makes contributions to software security and represents a crucial step toward preventing the misuse of LLMs for malicious purposes. Moreover, the Random Forest algorithm for binary malicious code classification in LLM-generated code achieved the best F$_{1}$ score of 94.92\% on the ChatGPT-generated dataset (with CodeT5 feature extraction technique). We also showed that the classification models exhibited poor performance on the dataset generated by Claude language model.

Zitieren

APA

Pehlivanoğlu, M. K., & Çoban, M. G. (2025). Detection of Malicious Codes Generated by Large Language Models: A Comparison of GPT-3.5, GPT-4o, Gemini, and Claude. International Journal of Information Security Science. https://doi.org/10.55859/ijiss.1634763

Vancouver

Pehlivanoğlu MK, Çoban MG. Detection of Malicious Codes Generated by Large Language Models: A Comparison of GPT-3.5, GPT-4o, Gemini, and Claude. International Journal of Information Security Science. 2025. doi:10.55859/ijiss.1634763

BibTeX
@article{Pehlivanolu2025,
  author = {Pehlivanoğlu, Meltem Kurt and Çoban, Murat Görkem},
  title = {Detection of Malicious Codes Generated by Large Language Models: A Comparison of GPT-3.5, GPT-4o, Gemini, and Claude},
  journal = {International Journal of Information Security Science},
  year = {2025},
  doi = {10.55859/ijiss.1634763}
}
Kostenlos starten →