EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

How Detectable Is Machine-Generated Text? A Preregistered Audit of Post-Training, Scale, and Task Effects

Abadila Alaktif, Meriyem Chergui, Gmira Faiq, Abdelkarim Ammoumou

Technologies · 02.11.2026 · 14 (11) · 718

DOI: 10.3390/technologies14110718 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Technologies, online erschienen am 02.11.2026. Verfasst von Abadila Alaktif und 3 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Why is machine-generated text detectable, and how much does the answer depend on the model’s scale, its post-training, and the writing task? We measure detectability with a validation-tuned linear probe on frozen embeddings from RoBERTa, a pretrained transformer language encoder, under a preregistered protocol in which every model stage receives an identical plain prompt, every output is exactly 256 tokens from a single attempt, and all evaluation rules were fixed before any score existed. Across nine model configurations evaluated in thirteen model–task cells (four Tulu-3 post-training stages under both rewriting and free generation; five Pythia sizes from 70M to 6.9B parameters), detectability at full instrument strength is near-perfect in every cell: ten of thirteen cells reach an area under the curve (AUROC) of 1.0, including the unaligned base model. With the instrument deliberately weakened to a 32-token view, exact ordered tests support a small increase in detectability across post-training stages in both tasks, but the rewrite and free-generation conditions do not differ (difference-in-differences −0.002, 95% CI [−0.033,0.029]), and no association between detectability and model scale is detected within the examined 70M–6.9B range (exact Spearman ρ=−0.1, p=0.475). At the level of individual texts, rewrites that copy more of their source are reliably harder to detect (clustered ρ=−0.27 to −0.53, all ten generators negative). The picture that emerges, for the examined configurations, domain, and instrument, is pervasive detectability with small, task-independent post-training effects, and it implies that detector evaluations need ceiling checks, matched prompts, and instrument-strength ladders before their comparisons can be trusted.

Zitieren

APA

Alaktif, A., Chergui, M., Faiq, G., & Ammoumou, A. (2026). How Detectable Is Machine-Generated Text? A Preregistered Audit of Post-Training, Scale, and Task Effects. Technologies, 14(11), 718. https://doi.org/10.3390/technologies14110718

Vancouver

Alaktif A, Chergui M, Faiq G, Ammoumou A. How Detectable Is Machine-Generated Text? A Preregistered Audit of Post-Training, Scale, and Task Effects. Technologies. 2026;14(11):718. doi:10.3390/technologies14110718

BibTeX
@article{Alaktif2026,
  author = {Alaktif, Abadila and Chergui, Meriyem and Faiq, Gmira and Ammoumou, Abdelkarim},
  title = {How Detectable Is Machine-Generated Text? A Preregistered Audit of Post-Training, Scale, and Task Effects},
  journal = {Technologies},
  year = {2026},
  volume = {14},
  number = {11},
  pages = {718},
  doi = {10.3390/technologies14110718}
}
Kostenlos starten →