EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Case Law as Data : Prompt Engineering Strategies for Case Outcome Extraction with Large Language Models in a Zero-Shot Setting

Guillaume Zambrano

Law, Technology and Humans · 27.11.2024 · 6 (3) · 80-101

DOI: 10.5204/lthj.3623 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Law, Technology and Humans, online erschienen am 27.11.2024. Verfasst von Guillaume Zambrano. Die Arbeit wurde bisher 10-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study explores the effectiveness of prompt optimization techniques for legal case outcome extraction using Large Language Models (LLMs). Two state-of-the-art LLMs, LLaMA3 70b and Mixtral 8x7b, are used in a zero-shot data extraction task on a diverse dataset of 400 French appellate court decisions. The results show that LLMs exhibit remarkable efficiency in extraction tasks. Our findings indicate that baseline prompts achieve high performance metrics, with a best F1 score of 0.980 and a worst F1 score of 0.853. Optimized prompts yield varying degrees of improvement, with a best F1 score of 0.994 and a worst F1 score of 0.912. While some optimized prompts demonstrate significant improvements, others exhibit minor or even negative changes. Our results suggest that the optimization process has a non-uniform impact on performance metrics, and the effectiveness of optimized prompts depends on the specific model and dataset being used. These results underscore the significance of prompt engineering in optimizing LLM performance for Legal Information Extraction and Litigation Analytics reliability.

Zitieren

APA

Zambrano, G. (2024). Case Law as Data : Prompt Engineering Strategies for Case Outcome Extraction with Large Language Models in a Zero-Shot Setting. Law, Technology and Humans, 6(3), 80-101. https://doi.org/10.5204/lthj.3623

Vancouver

Zambrano G. Case Law as Data : Prompt Engineering Strategies for Case Outcome Extraction with Large Language Models in a Zero-Shot Setting. Law, Technology and Humans. 2024;6(3):80-101. doi:10.5204/lthj.3623

BibTeX
@article{Zambrano2024,
  author = {Zambrano, Guillaume},
  title = {Case Law as Data : Prompt Engineering Strategies for Case Outcome Extraction with Large Language Models in a Zero-Shot Setting},
  journal = {Law, Technology and Humans},
  year = {2024},
  volume = {6},
  number = {3},
  pages = {80--101},
  doi = {10.5204/lthj.3623}
}
Kostenlos starten →