EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Mitigating adversarial manipulation in LLMs: a prompt-based approach to counter Jailbreak attacks (Prompt-G)

Bhagyajit Pingua, Deepak Murmu, Meenakshi Kandpal, Jyotirmayee Rautaray iD, Pranati Mishra, Rabindra Kumar Barik, Manob Jyoti Saikia iD

PeerJ Computer Science · 22.10.2024 · 10 · e2374

DOI: 10.7717/peerj-cs.2374 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 22.10.2024. Verfasst von Bhagyajit Pingua und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Sciences, Odisha University of Technology and Research, Bhubaneswar, Odisha, India. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Biomedical Sensors & Systems Lab.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have become transformative tools in areas like text generation, natural language processing, and conversational AI. However, their widespread use introduces security risks, such as jailbreak attacks, which exploit LLM’s vulnerabilities to manipulate outputs or extract sensitive information. Malicious actors can use LLMs to spread misinformation, manipulate public opinion, and promote harmful ideologies, raising ethical concerns. Balancing safety and accuracy require carefully weighing potential risks against benefits. Prompt Guarding (Prompt-G) addresses these challenges by using vector databases and embedding techniques to assess the credibility of generated text, enabling real-time detection and filtering of malicious content. We collected and analyzed a dataset of Self Reminder attacks to identify and mitigate jailbreak attacks, ensuring that the LLM generates safe and accurate responses. In various attack scenarios, Prompt-G significantly reduced jailbreak success rates and effectively identified prompts that caused confusion or distraction in the LLM. Integrating our model with Llama 2 13B chat reduced the attack success rate (ASR) to 2.08%. The source code is available at: https://doi.org/10.5281/zenodo.13501821 .

Schlagwörter

Stichwörter der Autor:innen

EmbeddingsJailbreak attacksLLMLarge language modelsLlama 2 13BVector databases

Autor:innen und Institutionen

  • School of Computer Sciences, Odisha University of Technology and Research, Bhubaneswar, Odisha, India.
  • School of Computer Applications, KIIT Deemed to be University, Bhubaneswar, Odisha, India.
  • Electrical and Computer Engineering, The University of Memphis, Memphis, TN, United States.
  • Biomedical Sensors & Systems Lab, The University of Memphis, Memphis, FL, United States.

ORCID-Profile: Jyotirmayee Rautaray, Manob Jyoti Saikia

Förderung

  • Biomedical Sensors & Systems Lab

Zitieren

APA

Pingua, B., Murmu, D., Kandpal, M., Rautaray, J., Mishra, P., Barik, R. K., & Saikia, M. J. (2024). Mitigating adversarial manipulation in LLMs: a prompt-based approach to counter Jailbreak attacks (Prompt-G). PeerJ Computer Science, 10, e2374. https://doi.org/10.7717/peerj-cs.2374

Vancouver

Pingua B, Murmu D, Kandpal M, Rautaray J, Mishra P, Barik RK, et al. Mitigating adversarial manipulation in LLMs: a prompt-based approach to counter Jailbreak attacks (Prompt-G). PeerJ Computer Science. 2024;10:e2374. doi:10.7717/peerj-cs.2374

BibTeX
@article{Pingua2024,
  author = {Pingua, Bhagyajit and Murmu, Deepak and Kandpal, Meenakshi and Rautaray, Jyotirmayee and Mishra, Pranati and Barik, Rabindra Kumar and Saikia, Manob Jyoti},
  title = {Mitigating adversarial manipulation in LLMs: a prompt-based approach to counter Jailbreak attacks (Prompt-G)},
  journal = {PeerJ Computer Science},
  year = {2024},
  volume = {10},
  pages = {e2374},
  doi = {10.7717/peerj-cs.2374}
}
Kostenlos starten →