Enhancing AI Feedback Systems: Lessons Learned from a RAG Implementation Study
Journal of University Teaching and Learning Practice · 16.11.2026 · 23 (4)
DOI: 10.53761/9vprqq24 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of University Teaching and Learning Practice, online erschienen am 16.11.2026. Verfasst von Christopher Hughes und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0).
Abstract (Originalsprache)
This study investigates the impact of Retrieval-Augmented Generation (RAG) on graduate students' perceptions of AI-generated feedback in a cyber-social learning environment. Using a non-equivalent cohort comparison quasi-experimental design, the research compared survey responses from Fall 2023 (n=96) and Spring 2024 (n=86) before and after implementing RAG. Participants engaged in iterative project cycles involving AI feedback, peer reviews, and instructor evaluations. Results revealed significant improvements in perceived feedback quality (p < .001, d = 0.41) and usefulness (p < .001, d = 0.51) following RAG implementation, while perceptions of actionability remained unchanged (p = .68, d = -0.05). Hierarchical regression analyses identified educational level as a positive predictor of quality and usefulness perceptions, whereas age and location negatively predicted actionability, and educational level positively predicted it. These findings suggest that RAG enhances the contextual relevance and perceived value of AI-generated feedback, particularly for advanced students. However, the lack of improvement in actionability highlights a persistent challenge in AI feedback systems. The study recommends integrating RAG with human support and developing targeted interventions to improve feedback literacy and implementation, especially for less advanced students and older learners.
Zitieren
APA
Hughes, C., Saini, A., Cope, B., & Kalantzis, M. (2026). Enhancing AI Feedback Systems: Lessons Learned from a RAG Implementation Study. Journal of University Teaching and Learning Practice, 23(4). https://doi.org/10.53761/9vprqq24
Vancouver
Hughes C, Saini A, Cope B, Kalantzis M. Enhancing AI Feedback Systems: Lessons Learned from a RAG Implementation Study. Journal of University Teaching and Learning Practice. 2026;23(4). doi:10.53761/9vprqq24
BibTeX
@article{Hughes2026,
author = {Hughes, Christopher and Saini, Akash and Cope, Bill and Kalantzis, Mary},
title = {Enhancing AI Feedback Systems: Lessons Learned from a RAG Implementation Study},
journal = {Journal of University Teaching and Learning Practice},
year = {2026},
volume = {23},
number = {4},
doi = {10.53761/9vprqq24}
}
EUVIMED