A Multimodal Voice Phishing Detection System Integrating Text and Audio Analysis
Applied Sciences · 18.10.2025 · 15 (20) · 11170
DOI: 10.3390/app152011170 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 18.10.2025. Verfasst von Jiwon Kim und 4 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Culture, Sports and Tourism.
Abstract (Originalsprache)
Voice phishing has emerged as a critical security threat, exploiting both linguistic manipulation and advances in synthetic speech technologies. Traditional keyword-based approaches often fail to capture contextual patterns or detect forged audio, limiting their effectiveness in real-world scenarios. To address this gap, we propose a multimodal voice phishing detection system that integrates text and audio analysis. The text module employs a KoBERT-based transformer classifier with self-attention interpretation, while the audio module leverages MFCC features and a CNN–BiLSTM classifier to identify synthetic speech. A fusion mechanism combines the outputs of both modalities, with experiments conducted on real-world call transcripts, phishing datasets, and synthetic voice corpora. The results demonstrate that the proposed system consistently achieves high values regarding the accuracy, precision, recall, and F1-score on validation data while maintaining robust performance in noisy and diverse real-call scenarios. Furthermore, attention-based interpretability enhances trustworthiness by revealing cross-token and discourse-level interaction patterns specific to phishing contexts. These findings highlight the potential of the proposed system as a reliable, explainable, and deployable solution for preventing the financial and social damage caused by voice phishing. Unlike prior studies limited to single-modality or shallow fusion, our work presents a fully integrated text–audio detection pipeline optimized for Korean real-world datasets and robust to noisy, multi-speaker conditions.
Förderung
- Ministry of Culture, Sports and Tourism
Zitieren
APA
Kim, J., Gu, S., Kim, Y., Lee, S., & Kang, C. (2025). A Multimodal Voice Phishing Detection System Integrating Text and Audio Analysis. Applied Sciences, 15(20), 11170. https://doi.org/10.3390/app152011170
Vancouver
Kim J, Gu S, Kim Y, Lee S, Kang C. A Multimodal Voice Phishing Detection System Integrating Text and Audio Analysis. Applied Sciences. 2025;15(20):11170. doi:10.3390/app152011170
BibTeX
@article{Kim2025,
author = {Kim, Jiwon and Gu, Seuli and Kim, Youngbeom and Lee, Sukwon and Kang, Changgu},
title = {A Multimodal Voice Phishing Detection System Integrating Text and Audio Analysis},
journal = {Applied Sciences},
year = {2025},
volume = {15},
number = {20},
pages = {11170},
doi = {10.3390/app152011170}
}
EUVIMED