Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis
Journal of Scientific Metrics and Evaluation · 10.02.2026 · 4 (1) · 12-25
DOI: 10.69821/josme.v4i1.25 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Scientific Metrics and Evaluation, online erschienen am 10.02.2026. Verfasst von Delia Consuegra-Herrera und 4 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Applied computing and artificial intelligence (AI) are increasingly implemented in higher education, yet evidence about their “impact” remains difficult to compare because studies use heterogeneous outcome metrics and inconsistent reporting. This study conducts a PRISMA 2020–guided systematic review and meta-analysis to evaluate how impact is measured in empirical research on AI/applied computing in higher education and to synthesize effects when data permit. Searches were performed in IEEE Xplore, ACM Digital Library, Scopus, SpringerLink, and Google Scholar (2020–2024). From 1,610 records, eight studies met inclusion criteria. Outcomes were reported as learning outcomes/engagement and technical AI model performance metrics, revealing substantial variability in operational definitions, instruments, and reporting completeness. Using a random-effects model, the pooled effect for learning outcomes and engagement was SMD = 3.32 (95% CI [2.77, 3.87]) with extreme heterogeneity (I² ≈ 99%), indicating limited comparability across studies. The findings suggest that the main barrier to cumulative evidence is weak metric coherence rather than a lack of measurable outcomes. Future work should separate learning outcomes from engagement, avoid treating technical model metrics as educational impact unless explicitly linked to student-level outcomes, and adopt minimum reporting standards to enable robust evaluation and synthesis.
Zitieren
APA
Consuegra-Herrera, D., Montero, K. D. C. R., Vásquez, M. M., Medina, A. J. S., & Vega, D. M. (2026). Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis. Journal of Scientific Metrics and Evaluation, 4(1), 12-25. https://doi.org/10.69821/josme.v4i1.25
Vancouver
Consuegra-Herrera D, Montero KDCR, Vásquez MM, Medina AJS, Vega DM. Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis. Journal of Scientific Metrics and Evaluation. 2026;4(1):12-25. doi:10.69821/josme.v4i1.25
BibTeX
@article{ConsuegraHerrera2026,
author = {Consuegra-Herrera, Delia and Montero, Katherin del Carmen Rodríguez and Vásquez, María Mitre and Medina, Antonio José Sucre and Vega, Dalila María},
title = {Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis},
journal = {Journal of Scientific Metrics and Evaluation},
year = {2026},
volume = {4},
number = {1},
pages = {12--25},
doi = {10.69821/josme.v4i1.25}
}
EUVIMED