EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching

Andrew Katz, Mitch Gerhardt, Michelle Soledad

International Journal of Qualitative Methods · 2024-01 · 23

DOI: 10.1177/16094069241293283 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Qualitative Methods, online erschienen am 14.11.2024. Verfasst von Andrew Katz und 2 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 84 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Science Foundation, Virginia Tech Academy of Data Science.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Feedback is a critical aspect of improvement. Unfortunately, when there is a lot of feedback from multiple sources, it can be difficult to distill the information into actionable insights. Consider student evaluations of teaching (SETs), which are important sources of feedback for educators. These evaluations can provide instructors with insights into what worked and did not during a semester. A collection of SETs can also be useful to administrators as signals for courses or entire programs. However, on a large scale as in high-enrollment courses or administrative records over several years, the number of SETs can render them difficult to analyze. In this paper, we discuss a novel method for analyzing SETs using natural language processing (NLP) and large language models (LLMs). We demonstrate the method by applying it to a corpus of 5000 SETs from a large public university. We show that the method can extract, embed, cluster, and summarize the SETs to identify the themes they contain. More generally, this work illustrates how to use NLP techniques and LLMs to generate a codebook for SETs. We conclude by discussing the implications of this method for analyzing SETs and other types of student writing in teaching and research settings.

Förderung

  • National Science Foundation
  • Virginia Tech Academy of Data Science

Zitieren

APA

Katz, A., Gerhardt, M., & Soledad, M. (2024). Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching. International Journal of Qualitative Methods, 23. https://doi.org/10.1177/16094069241293283

Vancouver

Katz A, Gerhardt M, Soledad M. Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching. International Journal of Qualitative Methods. 2024;23. doi:10.1177/16094069241293283

BibTeX
@article{Katz2024,
  author = {Katz, Andrew and Gerhardt, Mitch and Soledad, Michelle},
  title = {Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching},
  journal = {International Journal of Qualitative Methods},
  year = {2024},
  volume = {23},
  doi = {10.1177/16094069241293283}
}
Kostenlos starten →