EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis

Delia Consuegra-Herrera, Katherin del Carmen Rodríguez Montero, María Mitre Vásquez, Antonio José Sucre Medina, Dalila María Vega

Journal of Scientific Metrics and Evaluation · 10.02.2026 · 4 (1) · 12-25

DOI: 10.69821/josme.v4i1.25 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Scientific Metrics and Evaluation, online erschienen am 10.02.2026. Verfasst von Delia Consuegra-Herrera und 4 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Applied computing and artificial intelligence (AI) are increasingly implemented in higher education, yet evidence about their “impact” remains difficult to compare because studies use heterogeneous outcome metrics and inconsistent reporting. This study conducts a PRISMA 2020–guided systematic review and meta-analysis to evaluate how impact is measured in empirical research on AI/applied computing in higher education and to synthesize effects when data permit. Searches were performed in IEEE Xplore, ACM Digital Library, Scopus, SpringerLink, and Google Scholar (2020–2024). From 1,610 records, eight studies met inclusion criteria. Outcomes were reported as learning outcomes/engagement and technical AI model performance metrics, revealing substantial variability in operational definitions, instruments, and reporting completeness. Using a random-effects model, the pooled effect for learning outcomes and engagement was SMD = 3.32 (95% CI [2.77, 3.87]) with extreme heterogeneity (I² ≈ 99%), indicating limited comparability across studies. The findings suggest that the main barrier to cumulative evidence is weak metric coherence rather than a lack of measurable outcomes. Future work should separate learning outcomes from engagement, avoid treating technical model metrics as educational impact unless explicitly linked to student-level outcomes, and adopt minimum reporting standards to enable robust evaluation and synthesis.

Zitieren

APA

Consuegra-Herrera, D., Montero, K. D. C. R., Vásquez, M. M., Medina, A. J. S., & Vega, D. M. (2026). Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis. Journal of Scientific Metrics and Evaluation, 4(1), 12-25. https://doi.org/10.69821/josme.v4i1.25

Vancouver

Consuegra-Herrera D, Montero KDCR, Vásquez MM, Medina AJS, Vega DM. Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis. Journal of Scientific Metrics and Evaluation. 2026;4(1):12-25. doi:10.69821/josme.v4i1.25

BibTeX
@article{ConsuegraHerrera2026,
  author = {Consuegra-Herrera, Delia and Montero, Katherin del Carmen Rodríguez and Vásquez, María Mitre and Medina, Antonio José Sucre and Vega, Dalila María},
  title = {Evaluating AI in Higher Education: Metrics, Evidence Quality, and Meta-Analytic Synthesis},
  journal = {Journal of Scientific Metrics and Evaluation},
  year = {2026},
  volume = {4},
  number = {1},
  pages = {12--25},
  doi = {10.69821/josme.v4i1.25}
}
Kostenlos starten →