EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Automatic Scoring System for English Writing Based on Natural Language Processing: Assessment of Accuracy and Educational Effect

Yuqing Cui

Forum for Linguistic Studies · 06.12.2024 · 6 (6) · 222-237

DOI: 10.30564/fls.v6i6.7135 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Forum for Linguistic Studies, online erschienen am 06.12.2024. Verfasst von Yuqing Cui. Die Arbeit wurde bisher 7-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study investigates the effectiveness and educational impact of a novel NLP-based English writing auto-scoring system. Utilizing advanced machine learning techniques, including BERT and Graph Neural Networks, the system demonstrates high consistency with human raters (Quadratic Weighted Kappa of 0.92) across multiple dimensions of writing quality. A longitudinal study involving 500 students over a 16-week semester revealed significant improvements in writing abilities, with the most substantial gains observed in grammar and mechanics (28.5% increase) and organization and structure (23.7% increase). Through comprehensive system evaluation using multiple metrics, including Adjacent Agreement Rate and Root Mean Square Error, our system consistently outperformed existing baseline approaches, including commercial off-the-shelf solutions. The implementation of our system significantly enhanced teacher efficiency, reducing essay grading time by 62% and increasing time for individualized feedback by 45%. The system's architecture integrates cutting-edge NLP technologies with a user-friendly interface, facilitating real-time feedback and adaptive assessment capabilities. Our evaluation framework encompasses both technical accuracy and educational effectiveness, addressing a critical gap in current literature. While the system shows limitations in assessing highly creative writing and faces potential risks of student gaming, its overall impact on writing instruction and assessment is overwhelmingly positive. The study demonstrates that NLP-based auto-scoring systems can effectively scale writing assessment, provide timely feedback, and potentially democratize access to high-quality writing instruction. These findings suggest a path toward more efficient, personalized, and equitable writing education.

Zitieren

APA

Cui, Y. (2024). Automatic Scoring System for English Writing Based on Natural Language Processing: Assessment of Accuracy and Educational Effect. Forum for Linguistic Studies, 6(6), 222-237. https://doi.org/10.30564/fls.v6i6.7135

Vancouver

Cui Y. Automatic Scoring System for English Writing Based on Natural Language Processing: Assessment of Accuracy and Educational Effect. Forum for Linguistic Studies. 2024;6(6):222-237. doi:10.30564/fls.v6i6.7135

BibTeX
@article{Cui2024,
  author = {Cui, Yuqing},
  title = {Automatic Scoring System for English Writing Based on Natural Language Processing: Assessment of Accuracy and Educational Effect},
  journal = {Forum for Linguistic Studies},
  year = {2024},
  volume = {6},
  number = {6},
  pages = {222--237},
  doi = {10.30564/fls.v6i6.7135}
}
Kostenlos starten →