EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The Limits of ChatGPT’s Pragmatic Interpretation: An Experimental Study of Conversational Implicature in Indonesian Language Conversations

Aris Badara, Muhammad Rifa'ie, Ruslan Yusoh, Ibrahim Salama

Rainbow: Journal of Literature, Linguistics and Culture Studies · 10.10.2026 · 15 (3) · 299-312

DOI: 10.15294/rainbow.v15i3.65667 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Rainbow: Journal of Literature, Linguistics and Culture Studies, online erschienen am 10.10.2026. Verfasst von Aris Badara und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study evaluates ChatGPT’s interpretative accuracy and contextual sensitivity in understanding conversational implicatures in Indonesian. A computational experimental design employed 80 minimal stimulus pairs, yielding 160 responses across five communication themes and four pragmatic phenomena. Each pair contained identical utterances presented in different situational contexts. Data were collected using GPT-6 at a ‘High’ reasoning level via Temporary Chat (unpersonalised). Two pragmatics experts independently assessed the responses using a 0–2 ordinal rubric. The analysis comprised Interpretation Accuracy (IA), Pairwise Joint Accuracy (PJA), descriptive comparisons between conditions, and qualitative examination of pragmatic inferences. IA reached 98.13% for Rater 1 and 91.25% for Rater 2, while PJA reached 96.25% and 82.50%, respectively. Exact agreement was 93.13%, with a weighted Cohen’s kappa of 0.332, indicating limitations in scoring consistency. According to Rater 2, pairwise accuracy was highest for indirect criticism or evaluation (95.00%) and lowest for indirect refusal (65.00%). Accuracy was descriptively higher in Condition A (96.25%) than in Condition B (86.25%). Qualitative analysis identified instances of inferences exceeding contextual support and insufficient use of relevant contextual information. These findings highlight the importance of evaluating both interpretative accuracy and contextual justification, while acknowledging the sensitivity of performance estimates to rater judgments.

Zitieren

APA

Badara, A., Rifa'ie, M., Yusoh, R., & Salama, I. (2026). The Limits of ChatGPT’s Pragmatic Interpretation: An Experimental Study of Conversational Implicature in Indonesian Language Conversations. Rainbow: Journal of Literature, Linguistics and Culture Studies, 15(3), 299-312. https://doi.org/10.15294/rainbow.v15i3.65667

Vancouver

Badara A, Rifa'ie M, Yusoh R, Salama I. The Limits of ChatGPT’s Pragmatic Interpretation: An Experimental Study of Conversational Implicature in Indonesian Language Conversations. Rainbow: Journal of Literature, Linguistics and Culture Studies. 2026;15(3):299-312. doi:10.15294/rainbow.v15i3.65667

BibTeX
@article{Badara2026,
  author = {Badara, Aris and Rifa'ie, Muhammad and Yusoh, Ruslan and Salama, Ibrahim},
  title = {The Limits of ChatGPT’s Pragmatic Interpretation: An Experimental Study of Conversational Implicature in Indonesian Language Conversations},
  journal = {Rainbow: Journal of Literature, Linguistics and Culture Studies},
  year = {2026},
  volume = {15},
  number = {3},
  pages = {299--312},
  doi = {10.15294/rainbow.v15i3.65667}
}
Kostenlos starten →