EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS

Mohammad O. Alhawarat, Ayman J. Alnsour, Khalil M. Abdelnaby, Mohammed A. F. Al-Husainy

Applied Sciences · 17.11.2026 · 16 (22) · 11382

DOI: 10.3390/app162211382 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 17.11.2026. Verfasst von Mohammad O. Alhawarat und 3 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Arabic sentiment benchmarks are widely used to rank models, yet how much of their residual error reflects genuine model failure rather than incorrect ground-truth labels is rarely audited directly. We report a reconciled, two-annotator audit of every error (n = 401) made by the best-performing model on the Hotel Arabic Reviews Dataset (HARD), with moderate item-level agreement (Cohen’s κ = 0.41–0.42), categorizing each as label noise, genuine model error, or irreducibly ambiguous. We find that 43.4% of residual errors (49.2% of false positives, 32.6% of false negatives) are attributable to incorrect ground truth. Instantiating the random classification noise model with this measurement implies, for any classifier whose errors are independent of the label noise (a conditional assumption we flag, not a demonstrated fact), a measurable-accuracy ceiling of about 98.35% on HARD derived from a lower-bound noise estimate, leaving at most roughly 2.11 points of headroom for future work. Against that ceiling we place a five-seed comparison of four ensemble-combination rules over four Arabic transformer backbones, in which stacking’s gain over the best single model is modest and not robust: 96.40% vs. 96.24% macro-F1, nominally p = 0.023 but not significant after multiple-comparison correction, and no ensemble rule reliably beats the best backbone. Replicating the full protocol on ASTD and ArSAS shows that neither the backbone ranking nor the ensemble benefit generalizes to informal-register data. We release the reconciled annotations, protocol, and evaluation code.

Zitieren

APA

Alhawarat, M. O., Alnsour, A. J., Abdelnaby, K. M., & Al-Husainy, M. A. F. (2026). Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS. Applied Sciences, 16(22), 11382. https://doi.org/10.3390/app162211382

Vancouver

Alhawarat MO, Alnsour AJ, Abdelnaby KM, Al-Husainy MAF. Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS. Applied Sciences. 2026;16(22):11382. doi:10.3390/app162211382

BibTeX
@article{Alhawarat2026,
  author = {Alhawarat, Mohammad O. and Alnsour, Ayman J. and Abdelnaby, Khalil M. and Al-Husainy, Mohammed A. F.},
  title = {Label Noise and the Accuracy Ceiling in Arabic Sentiment Benchmarks: A Reconciled Error Audit of HARD with Replication on ASTD and ArSAS},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {22},
  pages = {11382},
  doi = {10.3390/app162211382}
}
Kostenlos starten →