PUMA: Secure inference of LLaMA-7B in five minutes
Security and Safety · 2025 · 4 · 2025014
DOI: 10.1051/sands/2025014 ↗Kurz gefasst
Fachartikel in der Zeitschrift Security and Safety, online erschienen am 23.10.2025. Verfasst von Ye Dong und 10 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Transformer models (e.g., Bert and GPT) have shown their dominance in machine learning tasks. Many cloud companies have begun to provide services based on Transformer models, examples include translation and text-speech conversion. However, such a service inevitably requires access to the client’s data, which might contain sensitive information. Theoretically, running the services under secure multi-party computation (MPC) could protect clients’ privacy. However, current MPC frameworks are still limited in terms of model performance, efficiency, deployment, and functionality, especially when facing complex Transformer models. To this end, we propose an MPC framework P UMA to enable secure and efficient Transformer model inference. We first design high-quality approximations for the bottleneck functions in Transformers such as GELU and Softmax, reducing about 20 − 76% computation and communication costs than state-of-the-art works without performance drop. Then, we provide concrete instantiations for secure Embedding and LayerNorm. These implementations produce correct results and integrate compatible system architectures of cleartext Transformer models. Finally, we conducted extensive experiments on six popular benchmarks: text classification/generation/summarization/translation, audio-to-text, and image-to-text. Results show that P UMA can finish most tasks in several minutes, with comparable model performance (e.g., accuracy) as cleartext, and even evaluate LLaMA-7B in less than 5 minutes to generate 1 token.
Zitieren
APA
Dong, Y., Lu, W., Zheng, Y., Wu, H., Zhao, D., Tan, J., Huang, Z., Hong, C., Wei, T., Chen, W., & Zhou, J. (2025). PUMA: Secure inference of LLaMA-7B in five minutes. Security and Safety, 4, 2025014. https://doi.org/10.1051/sands/2025014
Vancouver
Dong Y, Lu W, Zheng Y, Wu H, Zhao D, Tan J, et al. PUMA: Secure inference of LLaMA-7B in five minutes. Security and Safety. 2025;4:2025014. doi:10.1051/sands/2025014
BibTeX
@article{Dong2025,
author = {Dong, Ye and Lu, Wen-Jie and Zheng, Yancheng and Wu, Haoqi and Zhao, Derun and Tan, Jin and Huang, Zhicong and Hong, Cheng and Wei, Tao and Chen, Wen-Guang and Zhou, Jianying},
title = {PUMA: Secure inference of LLaMA-7B in five minutes},
journal = {Security and Safety},
year = {2025},
volume = {4},
pages = {2025014},
doi = {10.1051/sands/2025014}
}
EUVIMED