EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CIPHER: Cybersecurity Intelligent Penetration-Testing Helper for Ethical Researcher

Derry Pratama iD, Naufal Suryanto iD, Andro Aprila Adiputra iD, Thi-Thu-Huong Le iD, Ahmada Yusril Kadiptya iD, Muhammad Iqbal iD, Howon Kim iD

Sensors · 26.10.2024 · 24 (21) · 6878

DOI: 10.3390/s24216878 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 26.10.2024. Verfasst von Derry Pratama und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Engineering, Pusan National University, Busan 46241, Republic of Korea. Hauptthemen laut MEDLINE: Computer Security. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 89 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: MSIT (Ministry of Science and ICT), Korea, ITRC (Information Technology Research Center) support program, IITP (Institute for Information & Communications Technology Planning & Evaluation).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Penetration testing, a critical component of cybersecurity, typically requires extensive time and effort to find vulnerabilities. Beginners in this field often benefit from collaborative approaches with the community or experts. To address this, we develop Cybersecurity Intelligent Penetration-testing Helper for Ethical Researchers (CIPHER), a large language model specifically trained to assist in penetration testing tasks as a chatbot. Unlike software development, penetration testing involves domain-specific knowledge that is not widely documented or easily accessible, necessitating a specialized training approach for AI language models. CIPHER was trained using over 300 high-quality write-ups of vulnerable machines, hacking techniques, and documentation of open-source penetration testing tools augmented in an expert response structure. Additionally, we introduced the Findings, Action, Reasoning, and Results (FARR) Flow augmentation, a novel method to augment penetration testing write-ups to establish a fully automated pentesting simulation benchmark tailored for large language models. This approach fills a significant gap in traditional cybersecurity Q&A benchmarks and provides a realistic and rigorous standard for evaluating LLM’s technical knowledge, reasoning capabilities, and practical utility in dynamic penetration testing scenarios. In our assessments, CIPHER achieved the best overall performance in providing accurate suggestion responses compared to other open-source penetration testing models of similar size and even larger state-of-the-art models like Llama 3 70B and Qwen1.5 72B Chat, particularly on insane difficulty machine setups. This demonstrates that the current capabilities of general large language models (LLMs) are insufficient for effectively guiding users through the penetration testing process. We also discuss the potential for improvement through scaling and the development of better benchmarks using FARR Flow augmentation results.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Computer SecurityHumansSoftwareArtificial IntelligenceResearch Personnel

Stichwörter der Autor:innen

AI penetration testing assistantLLM evaluationdomain specific LLMlarge language modelpenetration testingpentesting LLMvulnerabillity detection

Autor:innen und Institutionen

  • School of Computer Science and Engineering, Pusan National University, Busan 46241, Republic of Korea.
  • IoT Research Center, Pusan National University, Busan 46241, Republic of Korea.
  • Blockchain Platform Research Center, Pusan National University, Busan 46241, Republic of Korea.

ORCID-Profile: Derry Pratama, Naufal Suryanto, Andro Aprila Adiputra, Thi-Thu-Huong Le, Ahmada Yusril Kadiptya, Muhammad Iqbal, Howon Kim

Förderung

  • MSIT (Ministry of Science and ICT), Korea
  • ITRC (Information Technology Research Center) support program
  • IITP (Institute for Information & Communications Technology Planning & Evaluation)

Zitieren

APA

Pratama, D., Suryanto, N., Adiputra, A. A., Le, T. T., Kadiptya, A. Y., Iqbal, M., & Kim, H. (2024). CIPHER: Cybersecurity Intelligent Penetration-Testing Helper for Ethical Researcher. Sensors, 24(21), 6878. https://doi.org/10.3390/s24216878

Vancouver

Pratama D, Suryanto N, Adiputra AA, Le TT, Kadiptya AY, Iqbal M, et al. CIPHER: Cybersecurity Intelligent Penetration-Testing Helper for Ethical Researcher. Sensors. 2024;24(21):6878. doi:10.3390/s24216878

BibTeX
@article{Pratama2024,
  author = {Pratama, Derry and Suryanto, Naufal and Adiputra, Andro Aprila and Le, Thi-Thu-Huong and Kadiptya, Ahmada Yusril and Iqbal, Muhammad and Kim, Howon},
  title = {CIPHER: Cybersecurity Intelligent Penetration-Testing Helper for Ethical Researcher},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {21},
  pages = {6878},
  doi = {10.3390/s24216878}
}
Kostenlos starten →