EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

PUMA: Secure inference of LLaMA-7B in five minutes

Ye Dong, Wen-Jie Lu, Yancheng Zheng, Haoqi Wu, Derun Zhao, Jin Tan, Zhicong Huang, Cheng Hong, Tao Wei, Wen-Guang Chen, Jianying Zhou

Security and Safety · 2025 · 4 · 2025014

DOI: 10.1051/sands/2025014 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Security and Safety, online erschienen am 23.10.2025. Verfasst von Ye Dong und 10 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformer models (e.g., Bert and GPT) have shown their dominance in machine learning tasks. Many cloud companies have begun to provide services based on Transformer models, examples include translation and text-speech conversion. However, such a service inevitably requires access to the client’s data, which might contain sensitive information. Theoretically, running the services under secure multi-party computation (MPC) could protect clients’ privacy. However, current MPC frameworks are still limited in terms of model performance, efficiency, deployment, and functionality, especially when facing complex Transformer models. To this end, we propose an MPC framework P UMA to enable secure and efficient Transformer model inference. We first design high-quality approximations for the bottleneck functions in Transformers such as GELU and Softmax, reducing about 20 − 76% computation and communication costs than state-of-the-art works without performance drop. Then, we provide concrete instantiations for secure Embedding and LayerNorm. These implementations produce correct results and integrate compatible system architectures of cleartext Transformer models. Finally, we conducted extensive experiments on six popular benchmarks: text classification/generation/summarization/translation, audio-to-text, and image-to-text. Results show that P UMA can finish most tasks in several minutes, with comparable model performance (e.g., accuracy) as cleartext, and even evaluate LLaMA-7B in less than 5 minutes to generate 1 token.

Zitieren

APA

Dong, Y., Lu, W., Zheng, Y., Wu, H., Zhao, D., Tan, J., Huang, Z., Hong, C., Wei, T., Chen, W., & Zhou, J. (2025). PUMA: Secure inference of LLaMA-7B in five minutes. Security and Safety, 4, 2025014. https://doi.org/10.1051/sands/2025014

Vancouver

Dong Y, Lu W, Zheng Y, Wu H, Zhao D, Tan J, et al. PUMA: Secure inference of LLaMA-7B in five minutes. Security and Safety. 2025;4:2025014. doi:10.1051/sands/2025014

BibTeX
@article{Dong2025,
  author = {Dong, Ye and Lu, Wen-Jie and Zheng, Yancheng and Wu, Haoqi and Zhao, Derun and Tan, Jin and Huang, Zhicong and Hong, Cheng and Wei, Tao and Chen, Wen-Guang and Zhou, Jianying},
  title = {PUMA: Secure inference of LLaMA-7B in five minutes},
  journal = {Security and Safety},
  year = {2025},
  volume = {4},
  pages = {2025014},
  doi = {10.1051/sands/2025014}
}
Kostenlos starten →