Detecting and Evaluating Bias in Large Language Models: Concepts, Methods, and Challenges
Journal of Behavioral Data Science · 2026
DOI: 10.35566/jbds/gao ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Behavioral Data Science, 2026. Verfasst von Zu Gao und 2 weiteren Autor:innen. Die Arbeit nennt 71 Literaturangaben.
Abstract (Originalsprache)
Large Language Models (LLMs) are increasingly deployed in sensitive real-world contexts, yet concerns remain about their biases and the harms they can cause. Existing surveys mostly discuss sources of bias and mitigation techniques, but give less systematic attention to how bias in LLMs should be detected, measured, and reported. This survey addresses that gap. We present a structured review of methods for detecting and evaluating bias in LLMs. We first introduce the conceptual foundations, including representational versus allocational harms and taxonomies of bias. We then discuss how to design evaluations in practice: specifying measurement targets, choosing datasets and metrics, and reasoning about validity and reliability. Building on this, we review intrinsic methods that probe representations and likelihoods, and extrinsic methods that assess bias in classification, question answering, open-ended generation, and dialogue. We further highlight recent advances in counterfactual and certification-based evaluation, which aim to provide stronger guarantees on fairness metrics. Beyond English-centric settings, we survey cross-lingual and application-specific evaluations, intersectional bias analysis, and meta-level issues such as evaluator reliability, metric robustness, reproducibility, and governance. The review concludes by synthesizing best practices and offering a practitioner-oriented checklist, providing both a conceptual map and a practical toolkit for evaluating bias in LLMs.
Zitieren
APA
Gao, Z., Tong, L., & Zhang, Z. (2026). Detecting and Evaluating Bias in Large Language Models: Concepts, Methods, and Challenges. Journal of Behavioral Data Science. https://doi.org/10.35566/jbds/gao
Vancouver
Gao Z, Tong L, Zhang Z. Detecting and Evaluating Bias in Large Language Models: Concepts, Methods, and Challenges. Journal of Behavioral Data Science. 2026. doi:10.35566/jbds/gao
BibTeX
@article{Gao2026,
author = {Gao, Zu and Tong, Lingbo and Zhang, Zhiyong},
title = {Detecting and Evaluating Bias in Large Language Models: Concepts, Methods, and Challenges},
journal = {Journal of Behavioral Data Science},
year = {2026},
doi = {10.35566/jbds/gao}
}
EUVIMED