EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

LLaMA 3 vs. State-of-the-Art Large Language Models: Performance in Detecting Nuanced Fake News

Stefan Emil Repede, Remus Brad

Computers · 11.11.2024 · 13 (11) · 292

DOI: 10.3390/computers13110292 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers, online erschienen am 11.11.2024. Verfasst von Stefan Emil Repede und 1 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study investigates the effectiveness of a proposed version of Meta’s LLaMA 3 model in detecting fake claims across bilingual (English and Romanian) datasets, focusing on a multi-class approach beyond traditional binary classifications in order to better mimic real-world scenarios. The research employs a proposed version of the LLaMA 3 model, optimized for identifying nuanced categories such as “Mostly True” and “Mostly False”, and compares its performance against leading large language models (LLMs) including Open AI’s ChatGPT versions, Google’s Gemini, and similar LLaMA models. The analysis reveals that the proposed LLaMA 3 model consistently outperforms its base version and older LLaMA models, particularly in the Romanian dataset, achieving the highest accuracy of 39% and demonstrating superior capabilities in identifying nuanced claims, over all the compared large language models. However, the model’s performance across both languages highlights some challenges, with generally low accuracy and difficulties in handling ambiguous categories by all the LLMs. The study also underscores the impact of language and cultural context on model reliability, noting that even state-of-the-art models like ChatGPT 4.o and Gemini exhibit inconsistencies when applied to Romanian text and more than a binary true/false approach.

Zitieren

APA

Repede, S. E., & Brad, R. (2024). LLaMA 3 vs. State-of-the-Art Large Language Models: Performance in Detecting Nuanced Fake News. Computers, 13(11), 292. https://doi.org/10.3390/computers13110292

Vancouver

Repede SE, Brad R. LLaMA 3 vs. State-of-the-Art Large Language Models: Performance in Detecting Nuanced Fake News. Computers. 2024;13(11):292. doi:10.3390/computers13110292

BibTeX
@article{Repede2024,
  author = {Repede, Stefan Emil and Brad, Remus},
  title = {LLaMA 3 vs. State-of-the-Art Large Language Models: Performance in Detecting Nuanced Fake News},
  journal = {Computers},
  year = {2024},
  volume = {13},
  number = {11},
  pages = {292},
  doi = {10.3390/computers13110292}
}
Kostenlos starten →