Automated Text Annotation Using a Semi-Supervised Approach with Meta Vectorizer and Machine Learning Algorithms for Hate Speech Detection
Applied Sciences · 26.01.2024 · 14 (3) · 1078
DOI: 10.3390/app14031078 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 26.01.2024. Verfasst von Shoffan Saifullah und 5 weiteren Autor:innen. Die Arbeit wurde bisher 30-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Polish Ministry of Education and Science, PLGrid Infrastructure.
Abstract (Originalsprache)
Text annotation is an essential element of the natural language processing approaches. The manual annotation process performed by humans has various drawbacks, such as subjectivity, slowness, fatigue, and possibly carelessness. In addition, annotators may annotate ambiguous data. Therefore, we have developed the concept of automated annotation to get the best annotations using several machine-learning approaches. The proposed approach is based on an ensemble algorithm of meta-learners and meta-vectorizer techniques. The approach employs a semi-supervised learning technique for automated annotation to detect hate speech. This involves leveraging various machine learning algorithms, including Support Vector Machine (SVM), Decision Tree (DT), K-Nearest Neighbors (KNN), and Naive Bayes (NB), in conjunction with Word2Vec and TF-IDF text extraction methods. The annotation process is performed using 13,169 Indonesian YouTube comments data. The proposed model used a Stemming approach using data from Sastrawi and new data of 2245 words. Semi-supervised learning uses 5%, 10%, and 20% of labeled data compared to performing labeling based on 80% of the datasets. In semi-supervised learning, the model learns from the labeled data, which provides explicit information, and the unlabeled data, which offers implicit insights. This hybrid approach enables the model to generalize and make informed predictions even when limited labeled data is available (based on self-learning). Ultimately, this enhances its ability to handle real-world scenarios with scarce annotated information. In addition, the proposed method uses a variety of thresholds for matching words labeled with hate speech ranging from 0.6, 0.7, 0.8, to 0.9. The experiments indicated that the DT-TF-IDF model has the best accuracy value of 97.1% with a scenario of 5%:80%:0.9. However, several other methods have accuracy above 90%, such as SVM (TF-IDF and Word2Vec) and KNN (Word2Vec), based on both text extraction methods in several test scenarios.
Förderung
- Polish Ministry of Education and Science
- PLGrid Infrastructure
Zitieren
APA
Saifullah, S., Dreżewski, R., Dwiyanto, F. A., Aribowo, A. S., Fauziah, Y., & Cahyana, N. H. (2024). Automated Text Annotation Using a Semi-Supervised Approach with Meta Vectorizer and Machine Learning Algorithms for Hate Speech Detection. Applied Sciences, 14(3), 1078. https://doi.org/10.3390/app14031078
Vancouver
Saifullah S, Dreżewski R, Dwiyanto FA, Aribowo AS, Fauziah Y, Cahyana NH. Automated Text Annotation Using a Semi-Supervised Approach with Meta Vectorizer and Machine Learning Algorithms for Hate Speech Detection. Applied Sciences. 2024;14(3):1078. doi:10.3390/app14031078
BibTeX
@article{Saifullah2024,
author = {Saifullah, Shoffan and Dreżewski, Rafał and Dwiyanto, Felix Andika and Aribowo, Agus Sasmito and Fauziah, Yuli and Cahyana, Nur Heri},
title = {Automated Text Annotation Using a Semi-Supervised Approach with Meta Vectorizer and Machine Learning Algorithms for Hate Speech Detection},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {3},
pages = {1078},
doi = {10.3390/app14031078}
}
EUVIMED