EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party

Tadej Todorović, Andrej Flogie, Daniel Hari

ELOPE: English Language Overseas Perspectives and Enquiries · 29.07.2025 · 22 (1) · 19-34

DOI: 10.4312/elope.22.1.19-34 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ELOPE: English Language Overseas Perspectives and Enquiries, online erschienen am 29.07.2025. Verfasst von Tadej Todorović und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study explores the feasibility of using generative AI (ChatGPT, Gemini, and DeepSeek) to automate speech act annotation in Harold Pinter’s play The Birthday Party. Three chatbots – ChatGPT, Gemini, and DeepSeek – were tested under three scenarios varying in the amount of theoretical material provided. Each chatbot’s output was compared to a manually annotated reference via a Python script measuring classification accuracy. Scenario 2 produced the highest accuracy overall (75–82%), while Scenario 1 underperformed, owing to incorrect reliance on external typologies, and Scenario 3 showed signs of overfitting. ChatGPT o1 emerged as the most accurate model, achieving 82% accuracy in Scenario 2. The findings suggest that GenAI chatbots can serve as valuable preliminary annotators when good prompt-engineering and well-curated theoretical material are provided. Future research could extend this methodology to more context-dependent texts, further refining promptengineering strategies and exploring larger linguistic corpora.

Zitieren

APA

Todorović, T., Flogie, A., & Hari, D. (2025). Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party. ELOPE: English Language Overseas Perspectives and Enquiries, 22(1), 19-34. https://doi.org/10.4312/elope.22.1.19-34

Vancouver

Todorović T, Flogie A, Hari D. Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party. ELOPE: English Language Overseas Perspectives and Enquiries. 2025;22(1):19-34. doi:10.4312/elope.22.1.19-34

BibTeX
@article{Todorovi2025,
  author = {Todorović, Tadej and Flogie, Andrej and Hari, Daniel},
  title = {Generative AI in Pragmatics: Assessing the Accuracy of Automated Speech Act Classification in Pinter’s The Birthday Party},
  journal = {ELOPE: English Language Overseas Perspectives and Enquiries},
  year = {2025},
  volume = {22},
  number = {1},
  pages = {19--34},
  doi = {10.4312/elope.22.1.19-34}
}
Kostenlos starten →