JAMAL: A Multidimensional Benchmark for Arabic Commonsense Reasoning Across Life-Domains and Cognitive Axes
BenchCouncil Transactions on Benchmarks, Standards and Evaluations · 30.06.2026 · 6
DOI: 10.66834/byjwsd23 ↗Kurz gefasst
Fachartikel in der Zeitschrift BenchCouncil Transactions on Benchmarks, Standards and Evaluations, online erschienen am 30.06.2026. Verfasst von Basma Sayah und 4 weiteren Autor:innen. Die Arbeit nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Commonsense is a broad and multifaceted concept, making its evaluation a persistent challenge in natural language processing (NLP). This paper introduces JAMAL (Arabic for “Camel”), a multidimensional framework and benchmark for Arabic commonsense reasoning. JAMAL is structured along three complementary axes: (i) a life-domain axis comprising a taxonomy of 56 functional categories informed by the World Health Organization’s International Classification of Functioning, Disability, and Health (ICF), capturing diverse aspects of daily human experience; (ii) a cognitive axis organizing commonsense into three reasoning types: everyday situations, general knowledge, and problem-solving; and (iii) a cultural grounding axis distinguishing between universal, western/global, and Arabic-specific knowledge. To operationalize this framework, benchmark items are constructed using psycholinguistically inspired principles of constrained contextual prediction. We evaluate five Arabic language models using JAMAL and observe consistent differences in their performance across all axes. Notably, FANAR-27B achieves the strongest overall results among all evaluated models, outperforming FANAR-9B and smaller baselines. Overall, JAMAL provides a structured and interpretable benchmark for evaluating commonsense reasoning in Arabic, supporting the development of more robust language models through systematic analysis of their behavioral limitations.
Zitieren
APA
Sayah, B., Nehar, A., Cherroun, H., Bellaouar, S., & Alam, F. (2026). JAMAL: A Multidimensional Benchmark for Arabic Commonsense Reasoning Across Life-Domains and Cognitive Axes. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 6. https://doi.org/10.66834/byjwsd23
Vancouver
Sayah B, Nehar A, Cherroun H, Bellaouar S, Alam F. JAMAL: A Multidimensional Benchmark for Arabic Commonsense Reasoning Across Life-Domains and Cognitive Axes. BenchCouncil Transactions on Benchmarks, Standards and Evaluations. 2026;6. doi:10.66834/byjwsd23
BibTeX
@article{Sayah2026,
author = {Sayah, Basma and Nehar, Attia and Cherroun, Hadda and Bellaouar, Slimane and Alam, Firoj},
title = {JAMAL: A Multidimensional Benchmark for Arabic Commonsense Reasoning Across Life-Domains and Cognitive Axes},
journal = {BenchCouncil Transactions on Benchmarks, Standards and Evaluations},
year = {2026},
volume = {6},
doi = {10.66834/byjwsd23}
}
EUVIMED