EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

GPT-4 passes the bar exam

Daniel Martin Katz iD, Michael James Bommarito, Shang Gao, Pablo Arredondo

Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · 15.04.2024 · 382 (2270)

DOI: 10.1098/rsta.2023.0254 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, online erschienen am 26.02.2024. Verfasst von Daniel Martin Katz und 3 weiteren Autor:innen. Beteiligte Einrichtung: Illinois Tech, Chicago Kent College of Law, Chicago, IL, USA. Die Arbeit wurde bisher 227-mal zitiert (Crossref) und nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper, we experimentally evaluate the zero-shot performance of GPT-4 against prior generations of GPT on the entire uniform bar examination (UBE), including not only the multiple-choice multistate bar examination (MBE), but also the open-ended multistate essay exam (MEE) and multistate performance test (MPT) components. On the MBE, GPT-4 significantly outperforms both human test-takers and prior models, demonstrating a 26% increase over ChatGPT and beating humans in five of seven subject areas. On the MEE and MPT, which have not previously been evaluated by scholars, GPT-4 scores an average of 4.2/6.0 when compared with much lower scores for ChatGPT. Graded across the UBE components, in the manner in which a human test-taker would be, GPT-4 scores approximately 297 points, significantly in excess of the passing threshold for all UBE jurisdictions. These findings document not just the rapid and remarkable advance of large language model performance generally, but also the potential for such models to support the delivery of legal services in society. This article is part of the theme issue ‘A complexity science approach to law and governance’.

Schlagwörter

Stichwörter der Autor:innen

Bar ExamGPT-4large language modelslegal complexitylegal languagelegal services

Autor:innen und Institutionen

  • Illinois Tech, Chicago Kent College of Law, Chicago, IL, USA.
  • CodeX, The Stanford Center for Legal Informatics, Stanford, CA, USA.
  • Bucerius Law School, Hamburg, Germany.
  • 273 Ventures, LLC, USA.
  • Casetext, Inc., USA.

ORCID-Profile: Daniel Martin Katz

Zitieren

APA

Katz, D. M., Bommarito, M. J., Gao, S., & Arredondo, P. (2024). GPT-4 passes the bar exam. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 382(2270). https://doi.org/10.1098/rsta.2023.0254

Vancouver

Katz DM, Bommarito MJ, Gao S, Arredondo P. GPT-4 passes the bar exam. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2024;382(2270). doi:10.1098/rsta.2023.0254

BibTeX
@article{Katz2024,
  author = {Katz, Daniel Martin and Bommarito, Michael James and Gao, Shang and Arredondo, Pablo},
  title = {GPT-4 passes the bar exam},
  journal = {Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences},
  year = {2024},
  volume = {382},
  number = {2270},
  doi = {10.1098/rsta.2023.0254}
}
Kostenlos starten →