How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · 15.10.2025 · 8 (3) · 2319-2330
DOI: 10.1609/aies.v8i3.36718 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, online erschienen am 15.10.2025. Verfasst von Agrima Seth und 5 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).
Abstract (Originalsprache)
Representational bias in large language models (LLMs) has predominantly been measured through single-response interactions and has focused on Global North-centric identities like race and gender. We expand on that research by conducting a systematic audit of GPT-4 Turbo to reveal how deeply encoded representational biases are and how they extend to less-explored dimensions of identity. We prompt GPT-4 Turbo to generate over 7,200 stories about significant life events (such as weddings) in India, using prompts designed to encourage diversity to varying extents. Comparing the diversity of religious and caste representation in the outputs against the actual population distribution in India as recorded in census data, we quantify the presence and “stickiness” of representational bias in the LLM for religion and caste. We find that GPT-4 responses consistently overrepresent culturally dominant groups far beyond their statistical representation, despite prompts intended to encourage representational diversity. Our findings also suggest that representational bias in LLMs has a winner-takes-all quality that is more biased than the likely distribution bias in their training data, and repeated prompt-based nudges have limited and inconsistent efficacy in dislodging these biases. These results suggest that diversifying training data alone may not be sufficient to correct LLM bias, highlighting the need for more fundamental changes in model development.
Zitieren
APA
Seth, A., Choudhury, M., Sitaram, S., Toyama, K., Vashistha, A., & Bali, K. (2025). How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8(3), 2319-2330. https://doi.org/10.1609/aies.v8i3.36718
Vancouver
Seth A, Choudhury M, Sitaram S, Toyama K, Vashistha A, Bali K. How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 2025;8(3):2319-2330. doi:10.1609/aies.v8i3.36718
BibTeX
@article{Seth2025,
author = {Seth, Agrima and Choudhury, Monojit and Sitaram, Sunayana and Toyama, Kentaro and Vashistha, Aditya and Bali, Kalika},
title = {How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion},
journal = {Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society},
year = {2025},
volume = {8},
number = {3},
pages = {2319--2330},
doi = {10.1609/aies.v8i3.36718}
}
EUVIMED