Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party
ELOPE: English Language Overseas Perspectives and Enquiries · 29.07.2025 · 22 (1) · 19-34
DOI: 10.4312/elope.22.1.19-34 ↗Kurz gefasst
Fachartikel in der Zeitschrift ELOPE: English Language Overseas Perspectives and Enquiries, online erschienen am 29.07.2025. Verfasst von Tadej Todorović und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
This study explores the feasibility of using generative AI (ChatGPT, Gemini, and DeepSeek) to automate speech act annotation in Harold Pinter’s play The Birthday Party. Three chatbots – ChatGPT, Gemini, and DeepSeek – were tested under three scenarios varying in the amount of theoretical material provided. Each chatbot’s output was compared to a manually annotated reference via a Python script measuring classification accuracy. Scenario 2 produced the highest accuracy overall (75–82%), while Scenario 1 underperformed, owing to incorrect reliance on external typologies, and Scenario 3 showed signs of overfitting. ChatGPT o1 emerged as the most accurate model, achieving 82% accuracy in Scenario 2. The findings suggest that GenAI chatbots can serve as valuable preliminary annotators when good prompt-engineering and well-curated theoretical material are provided. Future research could extend this methodology to more context-dependent texts, further refining promptengineering strategies and exploring larger linguistic corpora.
Zitieren
APA
Todorović, T., Flogie, A., & Hari, D. (2025). Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party. ELOPE: English Language Overseas Perspectives and Enquiries, 22(1), 19-34. https://doi.org/10.4312/elope.22.1.19-34
Vancouver
Todorović T, Flogie A, Hari D. Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party. ELOPE: English Language Overseas Perspectives and Enquiries. 2025;22(1):19-34. doi:10.4312/elope.22.1.19-34
BibTeX
@article{Todorovi2025,
author = {Todorović, Tadej and Flogie, Andrej and Hari, Daniel},
title = {Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party},
journal = {ELOPE: English Language Overseas Perspectives and Enquiries},
year = {2025},
volume = {22},
number = {1},
pages = {19--34},
doi = {10.4312/elope.22.1.19-34}
}
EUVIMED