EUVIMEDEUVIMED entdecken
Mixed-Methods-StudieOpen Access · CC BY 4.0

A Mixed-Method Study on the Effectiveness of Using ChatGPT in Academic Writing and Students’ Perceived Experiences

Eka Apriani, Sholihatul Hamidah Daulay, Fitria Aprilia, Abdul Gafur Marzuki, Idi Warsah, Dadan Supardan, Muthmainnah

Journal of Language and Education · 31.03.2025 · 11 (1) · 26-45

DOI: 10.17323/jle.2025.17913 ↗

Kurz gefasst

Mixed-Methods-Studie in der Zeitschrift Journal of Language and Education, online erschienen am 31.03.2025. Verfasst von Eka Apriani und 6 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

The advent of artificial intelligence in education has brought forward tools like ChatGPT, which can potentially enhance students' academic writing abilities. However, there is limited empirical evidence examining its effectiveness and students' perceptions of its utility in academic contexts.

Purpose

This study aimed to measure the effect of using ChatGPT on students' academic writing abilities and to investigate students’ perceived experiences regarding the use of ChatGPT in their writing process.

Method

An explanatory mixed-method design was employed, incorporating a quantitative experiment followed by a qualitative investigation. The quantitative phase involved 102 fifth-semester students from an English education department at a university in Indonesia. These students were randomized into clusters based on their proximate writing test scores, resulting in two homogenous classes of 25 students each. These classes were then assigned to either an experimental group, which received 14 sessions using ChatGPT as a learning tool for academic writing, or a control group, which received 14 sessions using non-generative tools. Pre-tests and post-tests were administered to both groups. The qualitative phase involved interviews with 10 selected students from the experimental group to explore their perceived experiences with ChatGPT.

Results

The pre-test scores indicated homogeneity between the experimental and control groups, with scores of 57.15 and 56.35 respectively. After the intervention, the post-test scores revealed significant improvement in the experimental group, with an average score of 81.11 compared to 60.30 in the control group. Statistical analysis demonstrated a significant disparity between the two groups (p-value = 0.0000 < 0.05 for the experimental group and p-value = 0.067 > 0.05 for the control group), suggesting that the use of ChatGPT significantly enhanced students' academic writing abilities. The qualitative findings supported these results, with students reporting that ChatGPT facilitated idea generation, organization, and construction in their writing process.

Conclusion

The study concludes that ChatGPT significantly improves students' academic writing abilities, as evidenced by both quantitative and qualitative data. The tool's capacity to assist in the formulation and organization of ideas presents substantial potential for its use in academic research and writing. Given these findings, ChatGPT could be a valuable addition to the educational toolkit for enhancing academic writing skills.

Zitieren

APA

Apriani, E., Daulay, S. H., Aprilia, F., Marzuki, A. G., Warsah, I., Supardan, D., & Muthmainnah (2025). A Mixed-Method Study on the Effectiveness of Using ChatGPT in Academic Writing and Students’ Perceived Experiences. Journal of Language and Education, 11(1), 26-45. https://doi.org/10.17323/jle.2025.17913

Vancouver

Apriani E, Daulay SH, Aprilia F, Marzuki AG, Warsah I, Supardan D, et al. A Mixed-Method Study on the Effectiveness of Using ChatGPT in Academic Writing and Students’ Perceived Experiences. Journal of Language and Education. 2025;11(1):26-45. doi:10.17323/jle.2025.17913

BibTeX
@article{Apriani2025,
  author = {Apriani, Eka and Daulay, Sholihatul Hamidah and Aprilia, Fitria and Marzuki, Abdul Gafur and Warsah, Idi and Supardan, Dadan and Muthmainnah},
  title = {A Mixed-Method Study on the Effectiveness of Using ChatGPT in Academic Writing and Students’ Perceived Experiences},
  journal = {Journal of Language and Education},
  year = {2025},
  volume = {11},
  number = {1},
  pages = {26--45},
  doi = {10.17323/jle.2025.17913}
}
Kostenlos starten →