EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Zero-Shot Detection of IndoT5-Synthesized Indonesian Scientific Abstracts Using mDeBERTa v3

Aldo Syahputra, Aris Wahyu Murdiyanto, Ulfi Saidata Aesyi

Indonesian Journal of Data and Science · 31.07.2026 · 7 (2)

DOI: 10.56705/ijodas.v7i2.457 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Indonesian Journal of Data and Science, online erschienen am 31.07.2026. Verfasst von Aldo Syahputra und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human evaluators struggle to distinguish original scientific abstracts from AI-generated text, as AI-produced formal language appears neat and convincing; prior studies report reviewers correctly identify only 68% of AI-generated abstracts while misclassifying 14% of human texts. This study presents an exploratory, generator-specific evaluation of mDeBERTa v3 using zero-shot Natural Language Inference (NLI) classification, applied to Indonesian scientific abstracts synthesized via IndoT5-base-paraphrase rather than AI-generated text in general. A balanced 2,274-abstract dataset paired human abstracts (SINTA 3 journals) with IndoT5-base-paraphrase outputs as the AI class. Mann-Whitney U analysis on seven linguistic features revealed significant differences (p < 0.001) across all. A critical anomaly emerged: AI texts showed higher sentence-length variation (SD = 15.44) than human texts (SD = 7.99), contradicting the assumption that AI text is more uniform, attributable to context-window exhaustion in IndoT5 producing semantic hallucinations when synthesizing dense abstracts. Testing three NLI scenarios showed a single instruction targeting this fluctuation achieved the highest Recall (76.52%) but with 790 false positives among 1,137 human abstracts, limiting accuracy to 53.52%; added complexity further degraded AI-class recall due to vocabulary overlap. A Random Forest classifier trained on the same features achieved 91.21% accuracy (F1 = 0.9130), substantially outperforming the zero-shot approach and confirming the anomaly as a strong, learnable signal. These results indicate zero-shot NLI can partially track a generator's mechanical artifacts through a single targeted instruction, but remains insufficiently precise to separate machine-error fluctuation from natural human variation, and is not recommended for standalone academic-integrity screening without further refinement

Zitieren

APA

Syahputra, A., Murdiyanto, A. W., & Aesyi, U. S. (2026). Zero-Shot Detection of IndoT5-Synthesized Indonesian Scientific Abstracts Using mDeBERTa v3. Indonesian Journal of Data and Science, 7(2). https://doi.org/10.56705/ijodas.v7i2.457

Vancouver

Syahputra A, Murdiyanto AW, Aesyi US. Zero-Shot Detection of IndoT5-Synthesized Indonesian Scientific Abstracts Using mDeBERTa v3. Indonesian Journal of Data and Science. 2026;7(2). doi:10.56705/ijodas.v7i2.457

BibTeX
@article{Syahputra2026,
  author = {Syahputra, Aldo and Murdiyanto, Aris Wahyu and Aesyi, Ulfi Saidata},
  title = {Zero-Shot Detection of IndoT5-Synthesized Indonesian Scientific Abstracts Using mDeBERTa v3},
  journal = {Indonesian Journal of Data and Science},
  year = {2026},
  volume = {7},
  number = {2},
  doi = {10.56705/ijodas.v7i2.457}
}
Kostenlos starten →