An optimized BERT fine-tuned model using an artificial bee colony algorithm for automatic essay score prediction
PeerJ Computer Science · 24.09.2024 · 10 · e2191
DOI: 10.7717/peerj-cs.2191 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 24.09.2024. Verfasst von Ridha Hussein Chassab und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia.
Abstract (Originalsprache)
Background
The Automatic Essay Score (AES) prediction system is essential in education applications. The AES system uses various textural and grammatical features to investigate the exact score value for AES. The derived features are processed by various linear regressions and classifiers that require the learning pattern to improve the overall score. Issues Moreover, the classifiers face catastrophic forgetting problems, which maximizes computation complexity and reduce prediction accuracy. The forgetting problem can be resolved using the freezing mechanism; however, the mechanism can cause prediction errors.
Method
Therefore, this research proposes an optimized Bi-directional Encoder Representation from Transformation (BERT) by applying the Artificial Bee Colony algorithm (ABC) and Fine-Tuned Model (ABC-BERT-FTM) to solve the forgetting problem, which leads to higher prediction accuracy. Therefore, the ABC algorithm reduces the forgetting problem by selecting optimized network parameters.
Results
Two AES datasets, ASAP and ETS, were used to evaluate the performance of the optimized BERT of the AES system, and a high accuracy of up to 98.5% was achieved. Thus, based on the result, we can conclude that optimizing the BERT with a suitable meta-heuristic algorithm, such as the ABC algorithm, can resolve the forgetting problem, eventually increasing the AES system’s prediction accuracy.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia.
Förderung
- Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia
Zitieren
APA
Chassab, R. H., Zakaria, L. Q., & Tiun, S. (2024). An optimized BERT fine-tuned model using an artificial bee colony algorithm for automatic essay score prediction. PeerJ Computer Science, 10, e2191. https://doi.org/10.7717/peerj-cs.2191
Vancouver
Chassab RH, Zakaria LQ, Tiun S. An optimized BERT fine-tuned model using an artificial bee colony algorithm for automatic essay score prediction. PeerJ Computer Science. 2024;10:e2191. doi:10.7717/peerj-cs.2191
BibTeX
@article{Chassab2024,
author = {Chassab, Ridha Hussein and Zakaria, Lailatul Qadri and Tiun, Sabrina},
title = {An optimized BERT fine-tuned model using an artificial bee colony algorithm for automatic essay score prediction},
journal = {PeerJ Computer Science},
year = {2024},
volume = {10},
pages = {e2191},
doi = {10.7717/peerj-cs.2191}
}
EUVIMED