Comparative Analysis of Traditional and Modern NLP Techniques on the CoLA Dataset: From POS Tagging to Large Language Models
IEEE Open Journal of the Computer Society · 2025 · 6 · 248-260
DOI: 10.1109/ojcs.2025.3526712 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Open Journal of the Computer Society, 2025. Verfasst von Abdessamad Benlahbib und 3 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The task of classifying linguistic acceptability, exemplified by the CoLA (Corpus of Linguistic Acceptability) dataset, poses unique challenges for natural language processing (NLP) models. These challenges include distinguishing between subtle grammatical errors, understanding complex syntactic structures, and detecting semantic inconsistencies, all of which make the task difficult even for human annotators. In this article, we compare a range of techniques, from traditional methods such as Part-of-Speech (POS) tagging and feature extraction methods like CountVectorizer with Term Frequency-Inverse Document Frequency (TF-IDF) and N-grams, to modern embeddings such as FastText and Embeddings from Language Models (ELMo), as well as deep learning architectures like transformers and Large Language Models (LLMs). Our experiments show a clear improvement in performance as models evolve from traditional to more advanced approaches. Notably, state-of-the-art (SOTA) results were obtained by fine-tuning GPT-4o with extensive hyperparameter tuning, including experimenting with various epochs and batch sizes. This comparative analysis provides valuable insights into the relative strengths of each technique for identifying morphological, syntactic, and semantic violations, highlighting the effectiveness of LLMs in these tasks.
Zitieren
APA
Benlahbib, A., Boumhidi, A., Fahfouh, A., & Alami, H. (2025). Comparative Analysis of Traditional and Modern NLP Techniques on the CoLA Dataset: From POS Tagging to Large Language Models. IEEE Open Journal of the Computer Society, 6, 248-260. https://doi.org/10.1109/ojcs.2025.3526712
Vancouver
Benlahbib A, Boumhidi A, Fahfouh A, Alami H. Comparative Analysis of Traditional and Modern NLP Techniques on the CoLA Dataset: From POS Tagging to Large Language Models. IEEE Open Journal of the Computer Society. 2025;6:248-260. doi:10.1109/ojcs.2025.3526712
BibTeX
@article{Benlahbib2025,
author = {Benlahbib, Abdessamad and Boumhidi, Achraf and Fahfouh, Anass and Alami, Hamza},
title = {Comparative Analysis of Traditional and Modern NLP Techniques on the CoLA Dataset: From POS Tagging to Large Language Models},
journal = {IEEE Open Journal of the Computer Society},
year = {2025},
volume = {6},
pages = {248--260},
doi = {10.1109/ojcs.2025.3526712}
}
EUVIMED