A real-world test of artificial intelligence infiltration of a university examinations system: A “Turing Test” case study
PLOS ONE · 26.06.2024 · 19 (6) · e0305354
DOI: 10.1371/journal.pone.0305354 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 26.06.2024. Verfasst von Peter Scarfe und 3 weiteren Autor:innen. Beteiligte Einrichtung: School Psychology and Clinical Language Sciences (PCLS), University of Reading, Reading, United Kingdom. Hauptthemen laut MEDLINE: Artificial Intelligence, Educational Measurement, Students, COVID-19. Die Arbeit wurde bisher 98-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Engineering and Physical Sciences Research Council.
Abstract (Originalsprache)
The recent rise in artificial intelligence systems, such as ChatGPT, poses a fundamental problem for the educational sector. In universities and schools, many forms of assessment, such as coursework, are completed without invigilation. Therefore, students could hand in work as their own which is in fact completed by AI. Since the COVID pandemic, the sector has additionally accelerated its reliance on unsupervised ‘take home exams’. If students cheat using AI and this is undetected, the integrity of the way in which students are assessed is threatened. We report a rigorous, blind study in which we injected 100% AI written submissions into the examinations system in five undergraduate modules, across all years of study, for a BSc degree in Psychology at a reputable UK university. We found that 94% of our AI submissions were undetected. The grades awarded to our AI submissions were on average half a grade boundary higher than that achieved by real students. Across modules there was an 83.4% chance that the AI submissions on a module would outperform a random selection of the same number of real student submissions.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- School Psychology and Clinical Language Sciences (PCLS), University of Reading, Reading, United Kingdom.
- Department of Psychology, University of Essex, Colchester, United Kingdom.
ORCID-Profile: Peter Scarfe, Etienne Roesch
Förderung
- Engineering and Physical Sciences Research Council
Zitieren
APA
Scarfe, P., Watcham, K., Clarke, A., & Roesch, E. (2024). A real-world test of artificial intelligence infiltration of a university examinations system: A “Turing Test” case study. PLOS ONE, 19(6), e0305354. https://doi.org/10.1371/journal.pone.0305354
Vancouver
Scarfe P, Watcham K, Clarke A, Roesch E. A real-world test of artificial intelligence infiltration of a university examinations system: A “Turing Test” case study. PLOS ONE. 2024;19(6):e0305354. doi:10.1371/journal.pone.0305354
BibTeX
@article{Scarfe2024,
author = {Scarfe, Peter and Watcham, Kelly and Clarke, Alasdair and Roesch, Etienne},
title = {A real-world test of artificial intelligence infiltration of a university examinations system: A “Turing Test” case study},
journal = {PLOS ONE},
year = {2024},
volume = {19},
number = {6},
pages = {e0305354},
doi = {10.1371/journal.pone.0305354}
}
EUVIMED