Evaluating Bias in Low Resource MLMs: A Comprehensive Study on Measuring Gender Bias in BERT for Bangla
SUST Journal of Science and Technology (SUST JST) · 16.08.2026 · 35 (1)
DOI: 10.63512//sustjst.2025.1002 ↗Kurz gefasst
Fachartikel in der Zeitschrift SUST Journal of Science and Technology (SUST JST), online erschienen am 16.08.2026. Verfasst von Ahsan Habib und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This paper investigates gender–profession associations in Bengali BERT family models using a controlled masked language model (MLM) probing framework. We construct a systematic evaluation corpus of 64,800 sentences using a full factorial design over gender-denoting person words, profession terms, and sentence templates. Using prior-normalized log-probability and sigmoid based scoring, we measure how profession context influences the likelihood of gendered terms. Across three models (mBERT, BUET BanglaBERT, and Sagor Sarker BanglaBERT) we observe consistent variations in association strength, with a general tendency toward stronger alignment with male-denoting words, alongside notable exceptions across profession categories. These findings reflect learned distributional patterns rather than explicit rules. We further examine the applicability of an English originated bias measurement framework in Bengali, highlighting challenges in interpretation due to linguistic and data differences. Overall, this work provides a controlled methodology and empirical analysis for studying gender profession associations in low-resource language models.
Zitieren
APA
Habib, A., Meem, S. T., & Hridi, S. I. (2026). Evaluating Bias in Low Resource MLMs: A Comprehensive Study on Measuring Gender Bias in BERT for Bangla. SUST Journal of Science and Technology (SUST JST), 35(1). https://doi.org/10.63512//sustjst.2025.1002
Vancouver
Habib A, Meem ST, Hridi SI. Evaluating Bias in Low Resource MLMs: A Comprehensive Study on Measuring Gender Bias in BERT for Bangla. SUST Journal of Science and Technology (SUST JST). 2026;35(1). doi:10.63512//sustjst.2025.1002
BibTeX
@article{Habib2026,
author = {Habib, Ahsan and Meem, Sadia Tasnim and Hridi, Sadia Islam},
title = {Evaluating Bias in Low Resource MLMs: A Comprehensive Study on Measuring Gender Bias in BERT for Bangla},
journal = {SUST Journal of Science and Technology (SUST JST)},
year = {2026},
volume = {35},
number = {1},
doi = {10.63512//sustjst.2025.1002}
}
EUVIMED