Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice
Proceedings of the ACM on Software Engineering · 12.07.2024 · 1 (FSE) · 1819-1840
DOI: 10.1145/3660788 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 12.07.2024. Verfasst von Ranim Khojah und 3 weiteren Autor:innen. Die Arbeit wurde bisher 103-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0).
Abstract (Originalsprache)
Large Language Models (LLMs) are frequently discussed in academia and the general public as support tools for virtually any use case that relies on the production of text, including software engineering. Currently, there is much debate, but little empirical evidence, regarding the practical usefulness of LLM-based tools such as ChatGPT for engineers in industry. We conduct an observational study of 24 professional software engineers who have been using ChatGPT over a period of one week in their jobs, and qualitatively analyse their dialogues with the chatbot as well as their overall experience (as captured by an exit survey). We find that rather than expecting ChatGPT to generate ready-to-use software artifacts (e.g., code), practitioners more often use ChatGPT to receive guidance on how to solve their tasks or learn about a topic in more abstract terms. We also propose a theoretical framework for how the (i) purpose of the interaction, (ii) internal factors (e.g., the user’s personality), and (iii) external factors (e.g., company policy) together shape the experience (in terms of perceived usefulness and trust). We envision that our framework can be used by future research to further the academic discussion on LLM usage by software engineering practitioners, and to serve as a reference point for the design of future empirical LLM research in this domain.
Zitieren
APA
Khojah, R., Mohamad, M., Leitner, P., & Neto, F. G. D. O. (2024). Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice. Proceedings of the ACM on Software Engineering, 1(FSE), 1819-1840. https://doi.org/10.1145/3660788
Vancouver
Khojah R, Mohamad M, Leitner P, Neto FGDO. Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice. Proceedings of the ACM on Software Engineering. 2024;1(FSE):1819-1840. doi:10.1145/3660788
BibTeX
@article{Khojah2024,
author = {Khojah, Ranim and Mohamad, Mazen and Leitner, Philipp and Neto, Francisco Gomes de Oliveira},
title = {Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice},
journal = {Proceedings of the ACM on Software Engineering},
year = {2024},
volume = {1},
number = {FSE},
pages = {1819--1840},
doi = {10.1145/3660788}
}
EUVIMED