EUVIMEDEUVIMED entdecken
Crossref

Methodological foundations for artificial intelligence ‐driven survey question generation

Ted K. Mburu, Kangxuan Rong, Campbell J. McColley, Alexandra Werth

Journal of Engineering Education · 2025-07 · 114 (3)

DOI: 10.1002/jee.70012 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Engineering Education, online erschienen am 22.05.2025. Verfasst von Ted K. Mburu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: Cornell University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

This study investigates the use of large language models to create adaptive, contextually relevant survey questions, aiming to enhance data quality in educational research without limiting scalability.

Purpose

We provide step‐by‐step methods to develop a dynamic survey instrument, driven by artificial intelligence (AI), and introduce the Synthetic Question–Response Analysis (SQRA) framework, a methodology designed to help evaluate AI‐generated questions before deployment with human participants.

Design

We examine the questions generated by our survey instrument, as well as compare AI‐to‐AI, generated through our SQRA framework, with AI‐to‐human interactions. Activity theory provides a theoretical lens to examine the dynamic interactions between AI and participants, highlighting the mutual influence within the survey tool.

Results

We found that AI‐generated questions were contextually relevant and adaptable, successfully incorporating course‐specific references. However, issues such as redundant phrasing, double‐barreled questions, and jargon affected the clarity of the questions. Although the SQRA framework exhibited limitations in replicating human response variability, its iterative refinement process proved effective in improving question quality, reinforcing the utility of this approach for enhancing AI‐driven surveys.

Conclusions

While AI‐driven question generation can enhance the scalability and personalization of open‐ended survey prompts, more research is needed to establish best practices for high‐quality educational research. The SQRA framework demonstrated practical utility for prompt refinement and initial validation of AI‐generated survey content, but it is not capable of replicating human responses. We highlight the importance of iterative prompt engineering, ethical considerations, and the need for methodological advancements in the development of trustworthy AI‐driven survey instruments for educational research.

Förderung

  • Cornell University

Zitieren

APA

Mburu, T. K., Rong, K., McColley, C. J., & Werth, A. (2025). Methodological foundations for artificial intelligence ‐driven survey question generation. Journal of Engineering Education, 114(3). https://doi.org/10.1002/jee.70012

Vancouver

Mburu TK, Rong K, McColley CJ, Werth A. Methodological foundations for artificial intelligence ‐driven survey question generation. Journal of Engineering Education. 2025;114(3). doi:10.1002/jee.70012

BibTeX
@article{Mburu2025,
  author = {Mburu, Ted K. and Rong, Kangxuan and McColley, Campbell J. and Werth, Alexandra},
  title = {Methodological foundations for artificial intelligence ‐driven survey question generation},
  journal = {Journal of Engineering Education},
  year = {2025},
  volume = {114},
  number = {3},
  doi = {10.1002/jee.70012}
}
Kostenlos starten →