EUVIMEDEUVIMED entdecken
Crossref

CLEAR: Error Analysis via LLM-as-a-Judge Made Easy

Asaf Yehudai, Lilach Eden, Yotam Perlitz, Roy Bar-Haim, Michal Shmueli-Scheuer

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (48) · 41736-41738

DOI: 10.1609/aaai.v40i48.42398 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Asaf Yehudai und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The evaluation of Large Language Models (LLMs) increasingly relies on other LLMs acting as judges. However, current evaluation paradigms typically yield a single score or ranking, answering which model is better but not why. While essential for benchmarking, these top-level scores obscure the specific, actionable reasons behind a model's performance. To bridge this gap, we introduce CLEAR, an interactive, open-source package for LLM-based error analysis. CLEAR first generates per-instance textual feedback, then it creates a set of system-level error issues, and quantifies the prevalence of each identified issue. Our package also provides users with an interactive dashboard that allows for a comprehensive error analysis through aggregate visualizations, applies interactive filters to isolate specific issues or score ranges, and drills down to the individual instances that exemplify a particular behavioral pattern. We demonstrate CLEAR analysis for RAG and Math benchmarks, and showcase its utility through a user case study.

Zitieren

APA

Yehudai, A., Eden, L., Perlitz, Y., Bar-Haim, R., & Shmueli-Scheuer, M. (2026). CLEAR: Error Analysis via LLM-as-a-Judge Made Easy. Proceedings of the AAAI Conference on Artificial Intelligence, 40(48), 41736-41738. https://doi.org/10.1609/aaai.v40i48.42398

Vancouver

Yehudai A, Eden L, Perlitz Y, Bar-Haim R, Shmueli-Scheuer M. CLEAR: Error Analysis via LLM-as-a-Judge Made Easy. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(48):41736-41738. doi:10.1609/aaai.v40i48.42398

BibTeX
@article{Yehudai2026,
  author = {Yehudai, Asaf and Eden, Lilach and Perlitz, Yotam and Bar-Haim, Roy and Shmueli-Scheuer, Michal},
  title = {CLEAR: Error Analysis via LLM-as-a-Judge Made Easy},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {48},
  pages = {41736--41738},
  doi = {10.1609/aaai.v40i48.42398}
}
Kostenlos starten →