EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Learning to detect AI texts and learning the limits

Jiří Milička iD, Anna Marklová iD, Ondřej Drobil, Eva Pospíšilová iD

PLOS One · 2025

DOI: 10.1371/journal.pone.0333007 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Jiří Milička und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Linguistics, Faculty of Arts, Charles University, Prague, Czech Republic. Hauptthemen laut MEDLINE: Learning, Artificial Intelligence, Reading. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Univerzita Karlova v Praze, Grantová Agentura České Republiky, Ministerstvo Školství, Mládeže a Tělovýchovy.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study investigates whether individuals can learn to accurately discriminate between human-written and AI-produced texts when provided with immediate feedback, and if they can use this feedback to recalibrate their self-perceived competence. We also explore the specific criteria individuals rely upon when making these decisions, focusing on textual style and perceived readability. We used GPT-4o to generate several hundred texts across various genres and text types comparable to Koditex, a multi-register corpus of human-written texts. We then presented randomized text pairs to 254 Czech native speakers who identified which text was human-written and which was AI-generated. Participants were randomly assigned to two conditions: one receiving immediate feedback after each trial, the other receiving no feedback until experiment completion. We recorded accuracy in identification, confidence levels, response times, and judgments about text readability along with demographic data and participants’ engagement with AI technologies prior to the experiment. Participants receiving immediate feedback showed significant improvement in accuracy and confidence calibration. Participants initially held incorrect assumptions about AI-generated text features, including expectations about stylistic rigidity and readability. Notably, without feedback, participants made the most errors precisely when feeling most confident—an issue largely resolved among the feedback group. The ability to differentiate between human and AI-generated texts can be effectively learned through targeted training with explicit feedback, which helps correct misconceptions about AI stylistic features and readability, as well as potential other variables that were not explored, while facilitating more accurate self-assessment. This finding might be particularly important in educational contexts, since the ability to identify AI-generated content is highly desirable and, more importantly, false confidence in this domain can be harmful.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LearningArtificial IntelligenceReadingHumansFemaleMaleAdultYoung AdultComprehension

Autor:innen und Institutionen

  • Department of Linguistics, Faculty of Arts, Charles University, Prague, Czech Republic.
  • Institute of Czech Language and Theory of Communication, Faculty of Arts, Charles University, Prague, Czech Republic.

ORCID-Profile: Jiří Milička, Anna Marklová, Eva Pospíšilová

Förderung

  • Univerzita Karlova v Praze
  • Grantová Agentura České Republiky
  • Ministerstvo Školství, Mládeže a Tělovýchovy

Zitieren

APA

Milička, J., Marklová, A., Drobil, O., & Pospíšilová, E. (2025). Learning to detect AI texts and learning the limits. PLOS One. https://doi.org/10.1371/journal.pone.0333007

Vancouver

Milička J, Marklová A, Drobil O, Pospíšilová E. Learning to detect AI texts and learning the limits. PLOS One. 2025. doi:10.1371/journal.pone.0333007

BibTeX
@article{Milika2025,
  author = {Milička, Jiří and Marklová, Anna and Drobil, Ondřej and Pospíšilová, Eva},
  title = {Learning to detect AI texts and learning the limits},
  journal = {PLOS One},
  year = {2025},
  doi = {10.1371/journal.pone.0333007}
}
Kostenlos starten →