CLEAR: Error Analysis via LLM-as-a-Judge Made Easy
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (48) · 41736-41738
DOI: 10.1609/aaai.v40i48.42398 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Asaf Yehudai und 4 weiteren Autor:innen.
Abstract (Originalsprache)
The evaluation of Large Language Models (LLMs) increasingly relies on other LLMs acting as judges. However, current evaluation paradigms typically yield a single score or ranking, answering which model is better but not why. While essential for benchmarking, these top-level scores obscure the specific, actionable reasons behind a model's performance. To bridge this gap, we introduce CLEAR, an interactive, open-source package for LLM-based error analysis. CLEAR first generates per-instance textual feedback, then it creates a set of system-level error issues, and quantifies the prevalence of each identified issue. Our package also provides users with an interactive dashboard that allows for a comprehensive error analysis through aggregate visualizations, applies interactive filters to isolate specific issues or score ranges, and drills down to the individual instances that exemplify a particular behavioral pattern. We demonstrate CLEAR analysis for RAG and Math benchmarks, and showcase its utility through a user case study.
Zitieren
APA
Yehudai, A., Eden, L., Perlitz, Y., Bar-Haim, R., & Shmueli-Scheuer, M. (2026). CLEAR: Error Analysis via LLM-as-a-Judge Made Easy. Proceedings of the AAAI Conference on Artificial Intelligence, 40(48), 41736-41738. https://doi.org/10.1609/aaai.v40i48.42398
Vancouver
Yehudai A, Eden L, Perlitz Y, Bar-Haim R, Shmueli-Scheuer M. CLEAR: Error Analysis via LLM-as-a-Judge Made Easy. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(48):41736-41738. doi:10.1609/aaai.v40i48.42398
BibTeX
@article{Yehudai2026,
author = {Yehudai, Asaf and Eden, Lilach and Perlitz, Yotam and Bar-Haim, Roy and Shmueli-Scheuer, Michal},
title = {CLEAR: Error Analysis via LLM-as-a-Judge Made Easy},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {48},
pages = {41736--41738},
doi = {10.1609/aaai.v40i48.42398}
}
EUVIMED