EUVIMEDObjevte EUVIMED
Crossref

Methodological foundations for artificial intelligence ‐driven survey question generation

Ted K. Mburu, Kangxuan Rong, Campbell J. McColley, Alexandra Werth

Journal of Engineering Education · 2025-07 · 114 (3)

DOI: 10.1002/jee.70012 ↗

At a glance

Journal article in Journal of Engineering Education, published online on 2025-05-22. Written by Ted K. Mburu and 3 co-authors. Cited 9 times so far (Crossref); 47 references. Funded by: Cornell University.

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

Background

This study investigates the use of large language models to create adaptive, contextually relevant survey questions, aiming to enhance data quality in educational research without limiting scalability.

Purpose

We provide step‐by‐step methods to develop a dynamic survey instrument, driven by artificial intelligence (AI), and introduce the Synthetic Question–Response Analysis (SQRA) framework, a methodology designed to help evaluate AI‐generated questions before deployment with human participants.

Design

We examine the questions generated by our survey instrument, as well as compare AI‐to‐AI, generated through our SQRA framework, with AI‐to‐human interactions. Activity theory provides a theoretical lens to examine the dynamic interactions between AI and participants, highlighting the mutual influence within the survey tool.

Results

We found that AI‐generated questions were contextually relevant and adaptable, successfully incorporating course‐specific references. However, issues such as redundant phrasing, double‐barreled questions, and jargon affected the clarity of the questions. Although the SQRA framework exhibited limitations in replicating human response variability, its iterative refinement process proved effective in improving question quality, reinforcing the utility of this approach for enhancing AI‐driven surveys.

Conclusions

While AI‐driven question generation can enhance the scalability and personalization of open‐ended survey prompts, more research is needed to establish best practices for high‐quality educational research. The SQRA framework demonstrated practical utility for prompt refinement and initial validation of AI‐generated survey content, but it is not capable of replicating human responses. We highlight the importance of iterative prompt engineering, ethical considerations, and the need for methodological advancements in the development of trustworthy AI‐driven survey instruments for educational research.

Funding

  • Cornell University

Cite

APA

Mburu, T. K., Rong, K., McColley, C. J., & Werth, A. (2025). Methodological foundations for artificial intelligence ‐driven survey question generation. Journal of Engineering Education, 114(3). https://doi.org/10.1002/jee.70012

Vancouver

Mburu TK, Rong K, McColley CJ, Werth A. Methodological foundations for artificial intelligence ‐driven survey question generation. Journal of Engineering Education. 2025;114(3). doi:10.1002/jee.70012

BibTeX
@article{Mburu2025,
  author = {Mburu, Ted K. and Rong, Kangxuan and McColley, Campbell J. and Werth, Alexandra},
  title = {Methodological foundations for artificial intelligence ‐driven survey question generation},
  journal = {Journal of Engineering Education},
  year = {2025},
  volume = {114},
  number = {3},
  doi = {10.1002/jee.70012}
}
Start for free →