EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Prediction of Arabic Legal Rulings Using Large Language Models

Adel Ammar, Anis Koubaa, Bilel Benjdira, Omer Nacar, Serry Sibaee

Electronics · 15.02.2024 · 13 (4) · 764

DOI: 10.3390/electronics13040764 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 15.02.2024. Verfasst von Adel Ammar und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Prince Sultan University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the intricate field of legal studies, the analysis of court decisions is a cornerstone for the effective functioning of the judicial system. The ability to predict court outcomes helps judges during the decision-making process and equips lawyers with invaluable insights, enhancing their strategic approaches to cases. Despite its significance, the domain of Arabic court analysis remains under-explored. This paper pioneers a comprehensive predictive analysis of Arabic court decisions on a dataset of 10,813 commercial court real cases, leveraging the advanced capabilities of the current state-of-the-art large language models. Through a systematic exploration, we evaluate three prevalent foundational models (LLaMA-7b, JAIS-13b, and GPT-3.5-turbo) and three training paradigms: zero-shot, one-shot, and tailored fine-tuning. In addition, we assess the benefit of summarizing and/or translating the original Arabic input texts. This leads to a spectrum of 14 model variants, for which we offer a granular performance assessment with a series of different metrics (human assessment, GPT evaluation, ROUGE, and BLEU scores). We show that all variants of LLaMA models yield limited performance, whereas GPT-3.5-based models outperform all other models by a wide margin, surpassing the average score of the dedicated Arabic-centric JAIS model by 50%. Furthermore, we show that all scores except human evaluation are inconsistent and unreliable for assessing the performance of large language models on court decision predictions. This study paves the way for future research, bridging the gap between computational linguistics and Arabic legal analytics.

Förderung

  • Prince Sultan University

Zitieren

APA

Ammar, A., Koubaa, A., Benjdira, B., Nacar, O., & Sibaee, S. (2024). Prediction of Arabic Legal Rulings Using Large Language Models. Electronics, 13(4), 764. https://doi.org/10.3390/electronics13040764

Vancouver

Ammar A, Koubaa A, Benjdira B, Nacar O, Sibaee S. Prediction of Arabic Legal Rulings Using Large Language Models. Electronics. 2024;13(4):764. doi:10.3390/electronics13040764

BibTeX
@article{Ammar2024,
  author = {Ammar, Adel and Koubaa, Anis and Benjdira, Bilel and Nacar, Omer and Sibaee, Serry},
  title = {Prediction of Arabic Legal Rulings Using Large Language Models},
  journal = {Electronics},
  year = {2024},
  volume = {13},
  number = {4},
  pages = {764},
  doi = {10.3390/electronics13040764}
}
Kostenlos starten →