EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereotypes and Safeguards

Geng Liu, Li Feng, Carlo Alberto Bono, Songbo Yang, Mengxiao Zhu, Francesco Pierri

ACM Transactions on Intelligent Systems and Technology · 31.10.2026 · 17 (5) · 1-33

DOI: 10.1145/3819074 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Intelligent Systems and Technology, online erschienen am 23.07.2026. Verfasst von Geng Liu und 5 weiteren Autor:innen. Die Arbeit nennt 66 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent research has highlighted that assigning specific personas to large language models (LLMs) can significantly increase harmful content generation. However, limited attention has been given to persona-driven toxicity in non-Western contexts, particularly in Chinese-based LLMs. In this article, we perform a large-scale, cross-model analysis of refusal behavior and persona-driven toxicity amplification across four Chinese LLMs, leveraging a comprehensive dataset of over \(1{,}400{,}000\) generated texts. We identify significant disparities in persona-driven refusal behavior, including systematic gender differences in refusal triggering across the evaluated Chinese LLMs. Furthermore, we provide quantitative evidence of persona-driven toxicity amplification with respect to model default baselines. We show that this amplification—whose magnitude varies substantially across models—is driven by interactions across several factors, involving persona conditioning, prompting strategy, target social group, and model-specific safety mechanisms. Leveraging model-specific regression analyses, we systematically characterize how persona categories, target social groups, and prompt templates independently and jointly shape both refusal behavior and output toxicity. As a complementary case study, we further explore an iterative, evaluator-guided mitigation strategy based on model feedback with an external LLM evaluator, demonstrating that highly toxic outputs can be substantially reduced without costly model retraining. Overall, our findings highlight the importance of culturally contextualized safety evaluations for Chinese-language LLMs and provide a structured framework for assessing persona-induced risks and exploratory mitigation strategies in LLM-generated content.

Zitieren

APA

Liu, G., Feng, L., Bono, C. A., Yang, S., Zhu, M., & Pierri, F. (2026). Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereotypes and Safeguards. ACM Transactions on Intelligent Systems and Technology, 17(5), 1-33. https://doi.org/10.1145/3819074

Vancouver

Liu G, Feng L, Bono CA, Yang S, Zhu M, Pierri F. Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereotypes and Safeguards. ACM Transactions on Intelligent Systems and Technology. 2026;17(5):1-33. doi:10.1145/3819074

BibTeX
@article{Liu2026,
  author = {Liu, Geng and Feng, Li and Bono, Carlo Alberto and Yang, Songbo and Zhu, Mengxiao and Pierri, Francesco},
  title = {Evaluating Chinese Large Language Models: The Influence of Persona Assignment on Stereotypes and Safeguards},
  journal = {ACM Transactions on Intelligent Systems and Technology},
  year = {2026},
  volume = {17},
  number = {5},
  pages = {1--33},
  doi = {10.1145/3819074}
}
Kostenlos starten →