EUVIMEDEUVIMED entdecken
Crossref

Comparative evaluation of Large Language Models using key metrics and emerging tools

Sarah McAvinue, Kapal Dev

Expert Systems · 2025-02 · 42 (2)

DOI: 10.1111/exsy.13719 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Expert Systems, online erschienen am 03.09.2024. Verfasst von Sarah McAvinue und 1 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 44 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This research involved designing and building an interactive generative AI application to conduct a comparative analysis of two advanced Large Language Models (LLMs), GPT‐4, and Claude 2, using Langsmith evaluation tools. The project was developed to explore the potential of LLMs in facilitating postgraduate course recommendations within a simulated environment at Munster Technological University (MTU). Designed for comparative analysis, the application enables testing of GPT‐4 and Claude 2 and can be hosted flexibly on either Amazon Web Services (AWS) or Azure. It utilizes advanced natural language processing and retrieval‐augmented generation (RAG) techniques to process proprietary data tailored to postgraduate needs. A key component of this research was the rigorous assessment of the LLMs using the Langsmith evaluation tool against both customized and standard benchmarks. The evaluation focused on metrics such as bias, safety, accuracy, cost, robustness, and latency. Additionally, adaptability covering critical features like language translation and internet access, was independently researched since the Langsmith tool does not evaluate this metric. This ensures a holistic assessment of the LLM's capabilities.

Zitieren

APA

McAvinue, S., & Dev, K. (2025). Comparative evaluation of Large Language Models using key metrics and emerging tools. Expert Systems, 42(2). https://doi.org/10.1111/exsy.13719

Vancouver

McAvinue S, Dev K. Comparative evaluation of Large Language Models using key metrics and emerging tools. Expert Systems. 2025;42(2). doi:10.1111/exsy.13719

BibTeX
@article{McAvinue2025,
  author = {McAvinue, Sarah and Dev, Kapal},
  title = {Comparative evaluation of Large Language Models using key metrics and emerging tools},
  journal = {Expert Systems},
  year = {2025},
  volume = {42},
  number = {2},
  doi = {10.1111/exsy.13719}
}
Kostenlos starten →