Same CEFR, Different Prompt: Instruction-Paraphrase Sensitivity in CEFR-Conditioned Language Model Writing
Applied Language Sciences · 01.11.2026
DOI: 10.65553/als.260113 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Language Sciences, online erschienen am 01.11.2026. Verfasst von Michele Carlo.
Abstract (Originalsprache)
This study tests whether equivalent instructions alter the linguistic features of writing generated at Common European Framework of Reference for Languages (CEFR) levels. Four instruction-tuned language models each generated responses at four CEFR levels across ten writing tasks, five equivalent instruction wordings, and five repetitions per condition (N = 4,000 texts). Twenty prespecified linguistic measures described vocabulary, grammar, discourse, and stance in each generated text. Requested level accounted for 27.8% of variation in these measures, whereas wording accounted for 0.1%; dispersion did not differ significantly across wordings. Classifier accuracy changed by at most 0.6 percentage points for an unseen wording, compared with an 11–16-point loss for an unseen task. Task effects remained large after adjustment for response length. Within these five paraphrases, task choice mattered more than instruction wording, supporting wording robustness under the tested conditions without establishing that generated texts validly represent human proficiency.
Zitieren
APA
Carlo, M. (2026). Same CEFR, Different Prompt: Instruction-Paraphrase Sensitivity in CEFR-Conditioned Language Model Writing. Applied Language Sciences. https://doi.org/10.65553/als.260113
Vancouver
Carlo M. Same CEFR, Different Prompt: Instruction-Paraphrase Sensitivity in CEFR-Conditioned Language Model Writing. Applied Language Sciences. 2026. doi:10.65553/als.260113
BibTeX
@article{Carlo2026,
author = {Carlo, Michele},
title = {Same CEFR, Different Prompt: Instruction-Paraphrase Sensitivity in CEFR-Conditioned Language Model Writing},
journal = {Applied Language Sciences},
year = {2026},
doi = {10.65553/als.260113}
}
EUVIMED