Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment Through Latent Acoustic Pattern Triggers
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (38) · 32015-32023
DOI: 10.1609/aaai.v40i38.40472 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Liang Lin und 11 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
As Audio Large Language Models (ALLMs) emerge as powerful tools for speech processing, their safety implications demand urgent attention. While considerable research has explored textual and vision safety, audio’s distinct characteristics present significant challenges. This paper first investigates: Is ALLM vulnerable to backdoor attacks exploiting acoustic triggers? In response to this issue, we introduce Hidden in the Noise (HIN), a novel backdoor attack framework designed to exploit subtle, audio-specific features. HIN applies acoustic modifications to raw audio waveforms, such as alterations to temporal dynamics and strategic injection of spectrally tailored noise. These changes introduce consistent patterns that an ALLM’s acoustic feature encoder captures, embedding robust triggers within the audio stream. To evaluate ALLM robustness against audio-feature-based triggers, we develop the AudioSafe benchmark, assessing nine distinct risk types. Extensive experiments on AudioSafe and three established safety datasets reveal critical vulnerabilities in existing ALLMs: (I) audio features like environment noise and speech rate variations achieve over 90% average attack success rate, (II) ALLMs exhibit significant sensitivity differences across acoustic features, particularly showing minimal response to volume as a trigger, and (III) poisoned sample inclusion causes only marginal loss curve fluctuations, highlighting the attack’s stealth.
Zitieren
APA
Lin, L., Yu, M., Luo, K., Zhang, Y., Peng, L., Wang, D., Tang, X., Zhang, Y., Yang, X., Zhou, Z., Wang, K., & Liu, Y. (2026). Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment Through Latent Acoustic Pattern Triggers. Proceedings of the AAAI Conference on Artificial Intelligence, 40(38), 32015-32023. https://doi.org/10.1609/aaai.v40i38.40472
Vancouver
Lin L, Yu M, Luo K, Zhang Y, Peng L, Wang D, et al. Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment Through Latent Acoustic Pattern Triggers. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(38):32015-32023. doi:10.1609/aaai.v40i38.40472
BibTeX
@article{Lin2026,
author = {Lin, Liang and Yu, Miao and Luo, Kaiwen and Zhang, Yibo and Peng, Lilan and Wang, Dexian and Tang, Xuehai and Zhang, Yuanhe and Yang, Xikang and Zhou, Zhenhong and Wang, Kun and Liu, Yang},
title = {Hidden in the Noise: Unveiling Backdoors in Audio LLMs Alignment Through Latent Acoustic Pattern Triggers},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {38},
pages = {32015--32023},
doi = {10.1609/aaai.v40i38.40472}
}
EUVIMED