Prediction of Arabic Legal Rulings Using Large Language Models
Electronics · 15.02.2024 · 13 (4) · 764
DOI: 10.3390/electronics13040764 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 15.02.2024. Verfasst von Adel Ammar und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Prince Sultan University.
Abstract (Originalsprache)
In the intricate field of legal studies, the analysis of court decisions is a cornerstone for the effective functioning of the judicial system. The ability to predict court outcomes helps judges during the decision-making process and equips lawyers with invaluable insights, enhancing their strategic approaches to cases. Despite its significance, the domain of Arabic court analysis remains under-explored. This paper pioneers a comprehensive predictive analysis of Arabic court decisions on a dataset of 10,813 commercial court real cases, leveraging the advanced capabilities of the current state-of-the-art large language models. Through a systematic exploration, we evaluate three prevalent foundational models (LLaMA-7b, JAIS-13b, and GPT-3.5-turbo) and three training paradigms: zero-shot, one-shot, and tailored fine-tuning. In addition, we assess the benefit of summarizing and/or translating the original Arabic input texts. This leads to a spectrum of 14 model variants, for which we offer a granular performance assessment with a series of different metrics (human assessment, GPT evaluation, ROUGE, and BLEU scores). We show that all variants of LLaMA models yield limited performance, whereas GPT-3.5-based models outperform all other models by a wide margin, surpassing the average score of the dedicated Arabic-centric JAIS model by 50%. Furthermore, we show that all scores except human evaluation are inconsistent and unreliable for assessing the performance of large language models on court decision predictions. This study paves the way for future research, bridging the gap between computational linguistics and Arabic legal analytics.
Förderung
- Prince Sultan University
Zitieren
APA
Ammar, A., Koubaa, A., Benjdira, B., Nacar, O., & Sibaee, S. (2024). Prediction of Arabic Legal Rulings Using Large Language Models. Electronics, 13(4), 764. https://doi.org/10.3390/electronics13040764
Vancouver
Ammar A, Koubaa A, Benjdira B, Nacar O, Sibaee S. Prediction of Arabic Legal Rulings Using Large Language Models. Electronics. 2024;13(4):764. doi:10.3390/electronics13040764
BibTeX
@article{Ammar2024,
author = {Ammar, Adel and Koubaa, Anis and Benjdira, Bilel and Nacar, Omer and Sibaee, Serry},
title = {Prediction of Arabic Legal Rulings Using Large Language Models},
journal = {Electronics},
year = {2024},
volume = {13},
number = {4},
pages = {764},
doi = {10.3390/electronics13040764}
}
EUVIMED