Application of large language models in legal queries: experiments with the artificial intelligence for legal assistance challenge
Logic Journal of the IGPL · 27.07.2026 · 34 (4)
DOI: 10.1093/jigpal/jzag043 ↗Kurz gefasst
Fachartikel in der Zeitschrift Logic Journal of the IGPL, online erschienen am 06.07.2026. Verfasst von Carlos A Martín und 4 weiteren Autor:innen. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: La Caixa Foundation, CajaCanarias Foundation.
Abstract (Originalsprache)
This study evaluates the applicability of generative artificial intelligence (AI), specifically Large Language Models (LLMs), in legal assistance tasks using the Artificial Intelligence for Legal Assistance challenge dataset, which includes 197 statutes, 2914 judicial cases, and 50 legal queries. The research compared conversational LLM tools with semantic retrieval systems based on dense embeddings. Results show that general-purpose tools (ChatGPT, NotebookLM) achieved poor discrimination, while embedding-based methods significantly improved accuracy: the OpenAI text-embedding-ada-002 model reached 46.43% retrieval accuracy, and a fine-tuned all-mpnet-base-v2 model improved from 18.9% to 31.87% (a 68.6% relative gain). These findings demonstrate that combining LLMs with semantic retrieval techniques enhances precision and scalability in legal information systems, offering a viable roadmap for developing domain-specific, efficient, and sustainable AI legal assistants.
Förderung
- La Caixa Foundation
- CajaCanarias Foundation
Zitieren
APA
Martín, C. A., Torres, J. M., Aguilar, R. M., Bacallado, M. A., & Alayón, S. (2026). Application of large language models in legal queries: experiments with the artificial intelligence for legal assistance challenge. Logic Journal of the IGPL, 34(4). https://doi.org/10.1093/jigpal/jzag043
Vancouver
Martín CA, Torres JM, Aguilar RM, Bacallado MA, Alayón S. Application of large language models in legal queries: experiments with the artificial intelligence for legal assistance challenge. Logic Journal of the IGPL. 2026;34(4). doi:10.1093/jigpal/jzag043
BibTeX
@article{Martn2026,
author = {Martín, Carlos A and Torres, Jesús M and Aguilar, Rosa M and Bacallado, Manuel A and Alayón, Silvia},
title = {Application of large language models in legal queries: experiments with the artificial intelligence for legal assistance challenge},
journal = {Logic Journal of the IGPL},
year = {2026},
volume = {34},
number = {4},
doi = {10.1093/jigpal/jzag043}
}
EUVIMED