Can we spot fake public comments generated by ChatGPT(-3.5, -4)?: Japanese stylometric analysis expose emulation created by one-shot learning
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0299031 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Wataru Zaitsu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Psychology, Mejiro University, Tokyo, Japan. Hauptthemen laut MEDLINE: Learning, Disinformation. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 14 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JSPS KAKENHI.
Abstract (Originalsprache)
Public comments are an important opinion for civic when the government establishes rules. However, recent AI can easily generate large quantities of disinformation, including fake public comments. We attempted to distinguish between human public comments and ChatGPT-generated public comments (including ChatGPT emulated that of humans) using Japanese stylometric analysis. Study 1 conducted multidimensional scaling (MDS) to compare 500 texts of five classes: Human public comments, GPT-3.5 and GPT-4 generated public comments only by presenting the titles of human public comments (i.e., zero-shot learning, GPT zero ), GPT-3.5 and GPT-4 emulated by presenting sentences of human public comments and instructing to emulate that (i.e., one-shot learning, GPT one ). The MDS results showed that the Japanese stylometric features of the public comments were completely different from those of the GPT zero -generated texts. Moreover, GPT one -generated public comments were closer to those of humans than those generated by GPT zero . In Study 2, the performance levels of the random forest (RF) classifier for distinguishing three classes (human, GPT zero , and GPT one texts). RF classifiers showed the best precision for the human public comments of approximately 90%, and the best precision for the fake public comments generated by GPT (GPT zero and GPT one ) was 99.5% by focusing on integrated next writing style features: phrase patterns, parts-of-speech (POS) bigram and trigram, and function words. Therefore, the current study concluded that we could discriminate between GPT-generated fake public comments and those written by humans at the present time.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Faculty of Psychology, Mejiro University, Tokyo, Japan.
- Institute of Interdisciplinary Research, Kyoto University of Advanced Science, Kyoto, Japan.
- Speech and Language Laboratory, Australian National University, Canberra, Australia.
- School of Informatics, Daido University, Aichi, Japan.
- College of Policy Science, Ritsumeikan University, Kyoto, Japan.
ORCID-Profile: Wataru Zaitsu, Shunichi Ishihara
Förderung
- JSPS KAKENHI
Zitieren
APA
Zaitsu, W., Jin, M., Ishihara, S., Tsuge, S., & Inaba, M. (2024). Can we spot fake public comments generated by ChatGPT(-3.5, -4)?: Japanese stylometric analysis expose emulation created by one-shot learning. PLOS ONE. https://doi.org/10.1371/journal.pone.0299031
Vancouver
Zaitsu W, Jin M, Ishihara S, Tsuge S, Inaba M. Can we spot fake public comments generated by ChatGPT(-3.5, -4)?: Japanese stylometric analysis expose emulation created by one-shot learning. PLOS ONE. 2024. doi:10.1371/journal.pone.0299031
BibTeX
@article{Zaitsu2024,
author = {Zaitsu, Wataru and Jin, Mingzhe and Ishihara, Shunichi and Tsuge, Satoru and Inaba, Mitsuyuki},
title = {Can we spot fake public comments generated by ChatGPT(-3.5, -4)?: Japanese stylometric analysis expose emulation created by one-shot learning},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0299031}
}
EUVIMED