ChatGPT for Textual Analysis? How to Use Generative LLMs in Accounting Research
Management Science · 2025-09 · 71 (9) · 7888-7906
DOI: 10.1287/mnsc.2023.03253 ↗Kurz gefasst
Fachartikel in der Zeitschrift Management Science, 2025-09. Verfasst von Ties de Kok. Die Arbeit wurde bisher 150-mal zitiert (Crossref) und nennt 14 Literaturangaben.
Abstract (Originalsprache)
Generative large language models (GLLMs), such as ChatGPT and GPT-4 by OpenAI, are emerging as powerful tools for textual analysis tasks in accounting research. GLLMs can solve any textual analysis task solvable using nongenerative methods as well as tasks previously only solvable using human coding. Whereas GLLMs are new and powerful, they also come with limitations and present new challenges that require care and due diligence. This paper highlights the applications of GLLMs for accounting research and compares them with existing methods. It also provides a framework on how to effectively use GLLMs by addressing key considerations, such as model selection, prompt engineering, and ensuring construct validity. In a case study, I demonstrate the capabilities of GLLMs by detecting nonanswers in earnings conference calls, a traditionally challenging task to automate. The new GPT method achieves an accuracy of 96% and reduces the nonanswer error rate by 70% relative to the existing Gow et al. (2021) method. Finally, I discuss the importance of addressing bias, replicability, and data sharing concerns when using GLLMs. Taken together, this paper provides researchers, reviewers, and editors with the knowledge and tools to effectively use and evaluate GLLMs for academic research. This paper was accepted by Eric So, accounting. Funding: Supported by the Foster School of Business – University of Washington. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2023.03253 .
Zitieren
APA
Kok, T. D. (2025). ChatGPT for Textual Analysis? How to Use Generative LLMs in Accounting Research. Management Science, 71(9), 7888-7906. https://doi.org/10.1287/mnsc.2023.03253
Vancouver
Kok TD. ChatGPT for Textual Analysis? How to Use Generative LLMs in Accounting Research. Management Science. 2025;71(9):7888-7906. doi:10.1287/mnsc.2023.03253
BibTeX
@article{Kok2025,
author = {Kok, Ties de},
title = {ChatGPT for Textual Analysis? How to Use Generative LLMs in Accounting Research},
journal = {Management Science},
year = {2025},
volume = {71},
number = {9},
pages = {7888--7906},
doi = {10.1287/mnsc.2023.03253}
}
EUVIMED