Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS
Applied Sciences · 17.11.2026 · 16 (22) · 11382
DOI: 10.3390/app162211382 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 17.11.2026. Verfasst von Mohammad O. Alhawarat und 3 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Arabic sentiment benchmarks are widely used to rank models, yet how much of their residual error reflects genuine model failure rather than incorrect ground-truth labels is rarely audited directly. We report a reconciled, two-annotator audit of every error (n = 401) made by the best-performing model on the Hotel Arabic Reviews Dataset (HARD), with moderate item-level agreement (Cohen’s κ = 0.41–0.42), categorizing each as label noise, genuine model error, or irreducibly ambiguous. We find that 43.4% of residual errors (49.2% of false positives, 32.6% of false negatives) are attributable to incorrect ground truth. Instantiating the random classification noise model with this measurement implies, for any classifier whose errors are independent of the label noise (a conditional assumption we flag, not a demonstrated fact), a measurable-accuracy ceiling of about 98.35% on HARD derived from a lower-bound noise estimate, leaving at most roughly 2.11 points of headroom for future work. Against that ceiling we place a five-seed comparison of four ensemble-combination rules over four Arabic transformer backbones, in which stacking’s gain over the best single model is modest and not robust: 96.40% vs. 96.24% macro-F1, nominally p = 0.023 but not significant after multiple-comparison correction, and no ensemble rule reliably beats the best backbone. Replicating the full protocol on ASTD and ArSAS shows that neither the backbone ranking nor the ensemble benefit generalizes to informal-register data. We release the reconciled annotations, protocol, and evaluation code.
Zitieren
APA
Alhawarat, M. O., Alnsour, A. J., Abdelnaby, K. M., & Al-Husainy, M. A. F. (2026). Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS. Applied Sciences, 16(22), 11382. https://doi.org/10.3390/app162211382
Vancouver
Alhawarat MO, Alnsour AJ, Abdelnaby KM, Al-Husainy MAF. Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS. Applied Sciences. 2026;16(22):11382. doi:10.3390/app162211382
BibTeX
@article{Alhawarat2026,
author = {Alhawarat, Mohammad O. and Alnsour, Ayman J. and Abdelnaby, Khalil M. and Al-Husainy, Mohammed A. F.},
title = {Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {22},
pages = {11382},
doi = {10.3390/app162211382}
}
EUVIMED