Combining Balancing Dataset and SentenceTransformers to Improve Short Answer Grading Performance
Applied Sciences · 25.05.2024 · 14 (11) · 4532
DOI: 10.3390/app14114532 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 25.05.2024. Verfasst von Maresha Caroline Wijanto und 1 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Land, Infrastructure, and Transport.
Abstract (Originalsprache)
Short-answer questions can encourage students to express their understanding. However, these answers can vary widely, leading to subjective assessments. Automatic short answer grading (ASAG) has become an important field of research. Recent studies have demonstrated a good performance using computationally expensive models. Additionally, available datasets are often unbalanced in terms of quantity. This research attempts to combine a simpler SentenceTransformers model with a balanced dataset, using prompt engineering in GPT to generate new sentences. Our recommended model also tries to fine-tune several hyperparameters to achieve optimal results. The research results show that the relatively small-sized all-distilroberta-v1 model can achieve a Pearson correlation value of 0.9586. The RMSE, F1-score, and accuracy score also provide better performances. This model is combined with the fine-tuning of hyperparameters, such as the use of gradient checkpointing, the split-size ratio for testing and training data, and the pre-processing steps. The best result is obtained when the new generated dataset from the GPT data augmentation is implemented. The newly generated dataset from GPT data augmentation achieves a cosine similarity score of 0.8 for the correct category. When applied to other datasets, our proposed method also shows an improved performance. Therefore, we conclude that a relatively small-sized model combined with the fine-tuning of the appropriate hyperparameters and a balanced dataset can provide performance results that surpass other models that require larger resources and are computationally expensive.
Förderung
- Ministry of Land, Infrastructure, and Transport
Zitieren
APA
Wijanto, M. C., & Yong, H. (2024). Combining Balancing Dataset and SentenceTransformers to Improve Short Answer Grading Performance. Applied Sciences, 14(11), 4532. https://doi.org/10.3390/app14114532
Vancouver
Wijanto MC, Yong H. Combining Balancing Dataset and SentenceTransformers to Improve Short Answer Grading Performance. Applied Sciences. 2024;14(11):4532. doi:10.3390/app14114532
BibTeX
@article{Wijanto2024,
author = {Wijanto, Maresha Caroline and Yong, Hwan-Seung},
title = {Combining Balancing Dataset and SentenceTransformers to Improve Short Answer Grading Performance},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {11},
pages = {4532},
doi = {10.3390/app14114532}
}
EUVIMED