Red Teaming Language Model Detectors with Language Models
Transactions of the Association for Computational Linguistics · 23.02.2024 · 12 · 174-189
DOI: 10.1162/tacl_a_00639 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 23.02.2024. Verfasst von Zhouxing Shi und 5 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially deceptive usage of LLMs, recent work has proposed algorithms to detect LLM-generated text and protect LLMs. In this paper, we investigate the robustness and reliability of these LLM detectors under adversarial attacks. We study two types of attack strategies: 1) replacing certain words in an LLM’s output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation. In both strategies, we leverage an auxiliary LLM to generate the word replacements or the instructional prompt. Different from previous works, we consider a challenging setting where the auxiliary LLM can also be protected by a detector. Experiments reveal that our attacks effectively compromise the performance of all detectors in the study with plausible generations, underscoring the urgent need to improve the robustness of LLM-generated text detection systems. Code is available at https://github.com/shizhouxing/LLM-Detector-Robustness.
Zitieren
APA
Shi, Z., Wang, Y., Yin, F., Chen, X., Chang, K., & Hsieh, C. (2024). Red Teaming Language Model Detectors with Language Models. Transactions of the Association for Computational Linguistics, 12, 174-189. https://doi.org/10.1162/tacl_a_00639
Vancouver
Shi Z, Wang Y, Yin F, Chen X, Chang K, Hsieh C. Red Teaming Language Model Detectors with Language Models. Transactions of the Association for Computational Linguistics. 2024;12:174-189. doi:10.1162/tacl_a_00639
BibTeX
@article{Shi2024,
author = {Shi, Zhouxing and Wang, Yihan and Yin, Fan and Chen, Xiangning and Chang, Kai-Wei and Hsieh, Cho-Jui},
title = {Red Teaming Language Model Detectors with Language Models},
journal = {Transactions of the Association for Computational Linguistics},
year = {2024},
volume = {12},
pages = {174--189},
doi = {10.1162/tacl_a_00639}
}
EUVIMED