SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection
AI · 22.09.2025 · 6 (9) · 239
DOI: 10.3390/ai6090239 ↗Kurz gefasst
Fachartikel in der Zeitschrift AI, online erschienen am 22.09.2025. Verfasst von Daichi Azuma und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Telecommunications Advancement Foundation (TAF).
Abstract (Originalsprache)
The automated detection of deceptive language is a crucial challenge in computational linguistics. This study provides a rigorous comparative analysis of three tiers of machine learning models for detecting instructed deception: traditional machine learning (SVM), fine-tuned discriminative models (BERT), and in-context learning with generalist Large Language Models (LLMs). Using the “cross-cultural deception detection” dataset, our findings reveal a clear performance hierarchy. While SVM performance is inconsistent, fine-tuned BERT models achieve substantially superior accuracy. Notably, a multilingual BERT model improves cross-topic accuracy on Spanish text to 90.14%, a gain of over 22 percentage points from its monolingual counterpart (67.20%). In contrast, modern LLMs perform poorly in zero-shot settings and fail to surpass the SVM baseline even with few-shot prompting, underscoring the effectiveness of task-specific fine-tuning. By transparently addressing the limitations of the solicited, low-stakes deception dataset, we establish a robust methodological baseline that clarifies the strengths of different modeling paradigms and informs future research into more complex, real-world deception phenomena.
Förderung
- Telecommunications Advancement Foundation (TAF)
Zitieren
APA
Azuma, D., Meléndez, R., Ptaszynski, M., Masui, F., Aslan, L., & Eronen, J. (2025). SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection. AI, 6(9), 239. https://doi.org/10.3390/ai6090239
Vancouver
Azuma D, Meléndez R, Ptaszynski M, Masui F, Aslan L, Eronen J. SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection. AI. 2025;6(9):239. doi:10.3390/ai6090239
BibTeX
@article{Azuma2025,
author = {Azuma, Daichi and Meléndez, René and Ptaszynski, Michal and Masui, Fumito and Aslan, Lara and Eronen, Juuso},
title = {SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection},
journal = {AI},
year = {2025},
volume = {6},
number = {9},
pages = {239},
doi = {10.3390/ai6090239}
}
EUVIMED