How Can You Tell if Your Large Language Model Could Be a Closet Antisemite? An Explainability-Based Audit Framework for Implicit Bias
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (45) · 38404-38412
DOI: 10.1609/aaai.v40i45.41181 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Arka Dutta und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Auditing large language models (LLMs) for biases is an ongoing and dynamic process, resembling a proverbial cat-and-mouse game. As researchers identify new vulnerabilities in LLMs, guardrails are updated to address them, prompting the need for innovative approaches to audit the increasingly fortified LLMs for biases. This paper makes three contributions. First, it introduces a scalable, explainable framework to measure biases against various identity groups across multiple open large language models. Second, it conducts a bias audit considering five well-known open LLMs and demonstrates their bias inclinations towards several historically disadvantaged groups. Our audit reveals disturbing antisemitic, Islamophobic, and xenophobic biases present in several well-known LLMs. Finally, we release a dataset of 1,000 probes curated under the supervision of an expert social scientist that can facilitate similar audits.
Zitieren
APA
Dutta, A., Fayyazi, R., Yang, S., & KhudaBukhsh, A. R. (2026). How Can You Tell if Your Large Language Model Could Be a Closet Antisemite? An Explainability-Based Audit Framework for Implicit Bias. Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), 38404-38412. https://doi.org/10.1609/aaai.v40i45.41181
Vancouver
Dutta A, Fayyazi R, Yang S, KhudaBukhsh AR. How Can You Tell if Your Large Language Model Could Be a Closet Antisemite? An Explainability-Based Audit Framework for Implicit Bias. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(45):38404-38412. doi:10.1609/aaai.v40i45.41181
BibTeX
@article{Dutta2026,
author = {Dutta, Arka and Fayyazi, Reza and Yang, Shanchieh and KhudaBukhsh, Ashiqur R.},
title = {How Can You Tell if Your Large Language Model Could Be a Closet Antisemite? An Explainability-Based Audit Framework for Implicit Bias},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {45},
pages = {38404--38412},
doi = {10.1609/aaai.v40i45.41181}
}
EUVIMED