EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Assessment Validity in the Age of Generative AI: A Natural Experiment

Håvar Brattli, Alexander Utne, Matthew Lynch

Informatics · 03.04.2026 · 13 (4) · 56

DOI: 10.3390/informatics13040056 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Informatics, online erschienen am 03.04.2026. Verfasst von Håvar Brattli und 2 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Universities play a dual role as sites of learning and as institutions that certify student competence through assessment. The rapid diffusion of generative artificial intelligence (GenAI) challenges this certification function by altering the conditions under which assessment evidence is produced. When powerful AI tools are widely available, grades may increasingly reflect a combination of individual understanding and external cognitive support rather than solely independent competence. This study examines how changes in assessment format interact with GenAI availability to reshape observable performance outcomes in higher education. Using exam grade data from a compulsory undergraduate course delivered over five years (2021–2025; N = 1066), the study exploits a naturally occurring change in assessment conditions as a natural experiment. From 2021 to 2024, the course was assessed using an AI-permissive take-home examination, while in 2025 the assessment shifted to an AI-restricted, supervised in-person examination. Course content, intended learning outcomes, grading criteria, examiner continuity, and the structural design of the examination tasks remained stable across cohorts. The results reveal a pronounced shift in grade distributions coinciding with the format change. Failure rates increased sharply in 2025, mid-range grades declined, and the proportion of top grades remained largely unchanged. Statistical analysis indicates a significant association between examination period and grade outcomes (χ2(5, N = 1066) = 60.62, p < 0.001), with a small-to-moderate effect size (Cramér’s V = 0.24), driven primarily by the increase in failing grades. These findings suggest that AI-permissive and AI-restricted assessment formats may not be measurement-equivalent under conditions of widespread GenAI use. The results raise concerns about construct validity and the credibility of grades as signals of independent competence, while also highlighting tensions between certification credibility and assessment authenticity.

Zitieren

APA

Brattli, H., Utne, A., & Lynch, M. (2026). Assessment Validity in the Age of Generative AI: A Natural Experiment. Informatics, 13(4), 56. https://doi.org/10.3390/informatics13040056

Vancouver

Brattli H, Utne A, Lynch M. Assessment Validity in the Age of Generative AI: A Natural Experiment. Informatics. 2026;13(4):56. doi:10.3390/informatics13040056

BibTeX
@article{Brattli2026,
  author = {Brattli, Håvar and Utne, Alexander and Lynch, Matthew},
  title = {Assessment Validity in the Age of Generative AI: A Natural Experiment},
  journal = {Informatics},
  year = {2026},
  volume = {13},
  number = {4},
  pages = {56},
  doi = {10.3390/informatics13040056}
}
Kostenlos starten →