EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection

Daichi Azuma, René Meléndez, Michal Ptaszynski, Fumito Masui, Lara Aslan, Juuso Eronen

AI · 22.09.2025 · 6 (9) · 239

DOI: 10.3390/ai6090239 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI, online erschienen am 22.09.2025. Verfasst von Daichi Azuma und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Telecommunications Advancement Foundation (TAF).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The automated detection of deceptive language is a crucial challenge in computational linguistics. This study provides a rigorous comparative analysis of three tiers of machine learning models for detecting instructed deception: traditional machine learning (SVM), fine-tuned discriminative models (BERT), and in-context learning with generalist Large Language Models (LLMs). Using the “cross-cultural deception detection” dataset, our findings reveal a clear performance hierarchy. While SVM performance is inconsistent, fine-tuned BERT models achieve substantially superior accuracy. Notably, a multilingual BERT model improves cross-topic accuracy on Spanish text to 90.14%, a gain of over 22 percentage points from its monolingual counterpart (67.20%). In contrast, modern LLMs perform poorly in zero-shot settings and fail to surpass the SVM baseline even with few-shot prompting, underscoring the effectiveness of task-specific fine-tuning. By transparently addressing the limitations of the solicited, low-stakes deception dataset, we establish a robust methodological baseline that clarifies the strengths of different modeling paradigms and informs future research into more complex, real-world deception phenomena.

Förderung

  • Telecommunications Advancement Foundation (TAF)

Zitieren

APA

Azuma, D., Meléndez, R., Ptaszynski, M., Masui, F., Aslan, L., & Eronen, J. (2025). SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection. AI, 6(9), 239. https://doi.org/10.3390/ai6090239

Vancouver

Azuma D, Meléndez R, Ptaszynski M, Masui F, Aslan L, Eronen J. SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection. AI. 2025;6(9):239. doi:10.3390/ai6090239

BibTeX
@article{Azuma2025,
  author = {Azuma, Daichi and Meléndez, René and Ptaszynski, Michal and Masui, Fumito and Aslan, Lara and Eronen, Juuso},
  title = {SVM, BERT, or LLM? A Comparative Study on Multilingual Instructed Deception Detection},
  journal = {AI},
  year = {2025},
  volume = {6},
  number = {9},
  pages = {239},
  doi = {10.3390/ai6090239}
}
Kostenlos starten →