EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Prompt Engineering as Applied-Pragmatic Activity: Development and Preliminary Evaluation of the Pragmatic Constraint Index in Management ESP

Dmitry Khramchenko

Journal of Language and Education · 30.06.2026 · 12 (2) · 51-76

DOI: 10.17323/jle.2026.39203 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Language and Education, online erschienen am 30.06.2026. Verfasst von Dmitry Khramchenko. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The increasing use of generative artificial intelligence (GenAI) in professional communication raises a new question for English for Specific Purposes (ESP) pedagogy: how should learners specify the linguistic and pragmatic conditions under which AI-generated texts are produced? Prompt engineering can be examined as a form of applied-pragmatic activity, yet the sociopragmatic content of learners’ prompts has received limited direct assessment.

Purpose

This study aims to develop and preliminarily evaluate the Pragmatic Constraint Index (PCI), a rubric for coding how explicitly learners’ prompts specify pragmatic and genre-related conditions, and to describe how management ESP students’ prompts differ before and after explicit instruction. The construct examined is prompt-focused specification of pragmatic conditions, not the acquisition of autonomous pragmatic competence.

Method

Forty-two management students in a C1-targeted ESP course at one Russian university completed two business correspondence tasks with GPT-5.2 in a one-group pre-test/post-test study without a control group; the two tasks used different scenarios. A two-week instructional intervention introduced the P.A.R.T. framework (Persona, Act, Register, Tone) and taught students to express metalinguistic directives. The 84 prompt chains (42 pre-test, 42 post-test; iterative prompts concatenated) were coded with the exploratory PCI (four categories, 0–12 points) and analyzed for word count, lexical density, and syntactic complexity; paired comparisons used Wilcoxon signed-rank tests with Holm correction. In a secondary analysis, two experts who were not told the phase or the PCI scores rated 20 AI-generated business messages (10 pre-test, 10 post-test) written by 20 different students. Model-level reproducibility is not claimed.

Results

Median prompt length increased from 23 to 85 words and median PCI from 2 to 10 points (Z = 5.65, Holm-adjusted p < .001, r = .87). The metalinguistic-directives category showed the largest mean increase (0.1 to 2.5). In the 20-output subsample, post-test messages received higher expert ratings for mitigation of face-threatening acts (U = 5.0, Holm-adjusted p = .002, r = .76). Pooled across the two phases, PCI and composite output ratings were correlated (Spearman’s r_s = .78), but neither within-phase correlation was statistically significant (pre-test r_s = .38; post-test r_s = .44; n = 10 each); the pooled coefficient therefore largely reflects the difference between phases and is not evidence of independent predictive validity. Qualitative examples showed more explicit sociopragmatic instructions in post-test prompts, alongside chains containing conflicting directives.

Conclusion

After instruction, students produced longer prompts with more explicit metalinguistic and genre-related constraints, which indicates observed uptake of the taught directives. The single-group design, the different test scenarios, and the immediate post-test do not allow causal or longitudinal claims, and observed uptake of taught directives should not be equated with demonstrated sociopragmatic competence. The findings provide preliminary evidence that explicit prompting can be taught and coded in management ESP; the PCI is best regarded as a coding rubric for observable instructions rather than a validated measure. The proposed ‘Editor-in-Chief’ assessment idea remains a proposal for future testing with controlled designs and delayed transfer tasks.

Zitieren

APA

Khramchenko, D. (2026). Prompt Engineering as Applied-Pragmatic Activity: Development and Preliminary Evaluation of the Pragmatic Constraint Index in Management ESP. Journal of Language and Education, 12(2), 51-76. https://doi.org/10.17323/jle.2026.39203

Vancouver

Khramchenko D. Prompt Engineering as Applied-Pragmatic Activity: Development and Preliminary Evaluation of the Pragmatic Constraint Index in Management ESP. Journal of Language and Education. 2026;12(2):51-76. doi:10.17323/jle.2026.39203

BibTeX
@article{Khramchenko2026,
  author = {Khramchenko, Dmitry},
  title = {Prompt Engineering as Applied-Pragmatic Activity: Development and Preliminary Evaluation of the Pragmatic Constraint Index in Management ESP},
  journal = {Journal of Language and Education},
  year = {2026},
  volume = {12},
  number = {2},
  pages = {51--76},
  doi = {10.17323/jle.2026.39203}
}
Kostenlos starten →