Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges
Journal of Artificial Intelligence Research · 26.08.2024 · 80 · 1497-1523
DOI: 10.1613/jair.1.15865 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Artificial Intelligence Research, online erschienen am 26.08.2024. Verfasst von Emanuele La Malfa und 8 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref).
Abstract (Originalsprache)
Some of the most powerful language models currently are proprietary systems, accessible only via (typically restrictive) web or software programming interfaces. This is the LanguageModels-as-a-Service (LMaaS) paradigm. In contrast with scenarios where full model access is available, as in the case of open-source models, such closed-off language models present specific challenges for evaluating, benchmarking, and testing them. This paper has two goals: on the one hand, we delineate how the aforementioned challenges act as impediments to the accessibility, reproducibility, reliability, and trustworthiness of LMaaS. We systematically examine the issues that arise from a lack of information about language models for each of these four aspects. We conduct a detailed analysis of existing solutions, put forth a number of recommendations, and highlight directions for future advancements. On the other hand, it serves as a synthesized overview of the licences and capabilities of the most popular LMaaS.
Zitieren
APA
Malfa, E. L., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Nazar, R., Cohn, A., Shadbolt, N., & Wooldridge, M. (2024). Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges. Journal of Artificial Intelligence Research, 80, 1497-1523. https://doi.org/10.1613/jair.1.15865
Vancouver
Malfa EL, Petrov A, Frieder S, Weinhuber C, Burnell R, Nazar R, et al. Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges. Journal of Artificial Intelligence Research. 2024;80:1497-1523. doi:10.1613/jair.1.15865
BibTeX
@article{Malfa2024,
author = {Malfa, Emanuele La and Petrov, Aleksandar and Frieder, Simon and Weinhuber, Christoph and Burnell, Ryan and Nazar, Raza and Cohn, Anthony and Shadbolt, Nigel and Wooldridge, Michael},
title = {Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges},
journal = {Journal of Artificial Intelligence Research},
year = {2024},
volume = {80},
pages = {1497--1523},
doi = {10.1613/jair.1.15865}
}
EUVIMED