EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Komparasi ChatGPT-4o, Claude Sonnet, dan Gemini Pro dalam Deteksi Phishing Berbahasa Indonesia Menggunakan Pendekatan Zero-Shot Prompting

Nur Rizky Aditya, Nur Hidayat, Yustian Servanda

Jurnal Sains dan Teknologi (JSIT) · 22.07.2026 · 6 (2) · 606-612

DOI: 10.47233/jsit.v6i2.4962 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Jurnal Sains dan Teknologi (JSIT), online erschienen am 22.07.2026. Verfasst von Nur Rizky Aditya und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Phishing attacks targeting Indonesian internet users continue to grow in volume and sophistication, exploiting local slang, domestic financial institution names, and cultural context, yet the unique linguistic and cultural characteristics of Bahasa Indonesia remain underexplored in the scientific literature on AI-based cybersecurity. This study aims to comparatively evaluate three state-of-the-art Large Language Models (LLMs)—ChatGPT-4o (OpenAI), Claude Sonnet (Anthropic), and Gemini Pro (Google)—for their effectiveness in detecting Indonesian-language phishing threats. Using a zero-shot prompting approach, a curated dataset of 100 samples (46 phishing, 54 legitimate) representing real-world Indonesian phishing modus operandi across banking, e-commerce, government, and social media contexts was evaluated under identical experimental conditions. Inter-annotator reliability was confirmed using Cohen’s Kappa (κ = 0.858), and model performance was measured using Accuracy, Precision, Recall, and F1-score. Results show that Gemini Pro achieved the best overall performance (F1 = 0.967, Accuracy = 97.00%), followed by ChatGPT-4o (F1 = 0.930, Precision = 100%) and Claude Sonnet (F1 = 0.909, Accuracy = 92.00%). All three models demonstrated strong capability as zero-shot phishing detectors, highlighting the potential of generative LLMs as accessible, training-free components for AI-based cybersecurity systems in Indonesian-language environments, and offering practical guidance for selecting LLM tools for phishing detection in Indonesia.

Zitieren

APA

Aditya, N. R., Hidayat, N., & Servanda, Y. (2026). Komparasi ChatGPT-4o, Claude Sonnet, dan Gemini Pro dalam Deteksi Phishing Berbahasa Indonesia Menggunakan Pendekatan Zero-Shot Prompting. Jurnal Sains dan Teknologi (JSIT), 6(2), 606-612. https://doi.org/10.47233/jsit.v6i2.4962

Vancouver

Aditya NR, Hidayat N, Servanda Y. Komparasi ChatGPT-4o, Claude Sonnet, dan Gemini Pro dalam Deteksi Phishing Berbahasa Indonesia Menggunakan Pendekatan Zero-Shot Prompting. Jurnal Sains dan Teknologi (JSIT). 2026;6(2):606-612. doi:10.47233/jsit.v6i2.4962

BibTeX
@article{Aditya2026,
  author = {Aditya, Nur Rizky and Hidayat, Nur and Servanda, Yustian},
  title = {Komparasi ChatGPT-4o, Claude Sonnet, dan Gemini Pro dalam Deteksi Phishing Berbahasa Indonesia Menggunakan Pendekatan Zero-Shot Prompting},
  journal = {Jurnal Sains dan Teknologi (JSIT)},
  year = {2026},
  volume = {6},
  number = {2},
  pages = {606--612},
  doi = {10.47233/jsit.v6i2.4962}
}
Kostenlos starten →