EUVIMEDEUVIMED entdecken
Crossref

Do source use features impact raters’ judgment of argumentation? An experimental study

Ping-Lin Chuang

Language Testing · 2025-01 · 42 (1) · 48-72

DOI: 10.1177/02655322241263629 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Language Testing, online erschienen am 31.07.2024. Verfasst von Ping-Lin Chuang. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 38 Literaturangaben. Gefördert von: Duolingo English Test Dissertation Research Awards in Language Assessment, TOEFL Grant for Doctoral Research in Language Assessment.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This experimental study explores how source use features impact raters’ judgment of argumentation in a second language (L2) integrated writing test. One hundred four experienced and novice raters were recruited to complete a rating task that simulated the scoring assignment of a local English Placement Test (EPT). Sixty written responses were adapted from essays written by EPT test-takers. These responses were crafted to reflect different conditions of source use features, namely source use quantity and quality. Rater scores were analyzed using the many-facet Rasch model and mixed two-way analyses of variance (ANOVAs) to examine how they are affected by source use features and rater experience. Results show that source use features impacted the argumentation scores assigned by raters. Paragraphs with more source text ideas that are better incorporated received the highest argumentation scores, and vice versa for those with limited, poorly integrated source information. Rater experience impacted scores but did not influence rater performance meaningfully. The findings of this study connect specific source use features with raters’ evaluation of argumentation, helping to further disentangle the relationships among examinee performance, rater decision, and task features of integrated argumentative writing tests. They also provide meaningful implications for writing assessment research and practices.

Förderung

  • Duolingo English Test Dissertation Research Awards in Language Assessment
  • TOEFL Grant for Doctoral Research in Language Assessment

Zitieren

APA

Chuang, P. (2025). Do source use features impact raters’ judgment of argumentation? An experimental study. Language Testing, 42(1), 48-72. https://doi.org/10.1177/02655322241263629

Vancouver

Chuang P. Do source use features impact raters’ judgment of argumentation? An experimental study. Language Testing. 2025;42(1):48-72. doi:10.1177/02655322241263629

BibTeX
@article{Chuang2025,
  author = {Chuang, Ping-Lin},
  title = {Do source use features impact raters’ judgment of argumentation? An experimental study},
  journal = {Language Testing},
  year = {2025},
  volume = {42},
  number = {1},
  pages = {48--72},
  doi = {10.1177/02655322241263629}
}
Kostenlos starten →