EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models

Ziniu Liu, Yue Han, Yang Song, Zhuwei Zhang, Aiping Li

Electronics · 07.03.2026 · 15 (5) · 1113

DOI: 10.3390/electronics15051113 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 07.03.2026. Verfasst von Ziniu Liu und 4 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Steering large language models (LLMs) toward desired behaviors while preserving privacy is a critical challenge in AI alignment. Existing differentially private (DP) steering methods, such as PSA, add high-dimensional noise that can severely degrade steering accuracy. We propose DP-JL, a novel approach that combines Johnson–Lindenstrauss (JL) random projection with differential privacy to reduce noise while maintaining formal privacy guarantees. DP-JL projects steering vectors into a lower-dimensional space (dimension k) before adding DP noise, reducing total noise magnitude from O(d) to O(k) where k≪d, while the privacy budget ε remains unchanged. We evaluate DP-JL on seven behavioral datasets with LLaMA-2-7B, Mistral-7B, Qwen2.5-7B, and Gemma-2-9B, alongside general capability benchmarks (MMLU, TruthfulQA). All accuracy values are measured on held-out test sets. Results show that DP-JL achieves: (1) up to 22.76 percentage points higher steering accuracy than PSA on the myopic-reward dataset (at fixed privacy budget ε≈0.22, δ=10−5); (2) 91.7% win rate on sycophancy with an average accuracy improvement of 3.01 percentage points; (3) systematic advantages in high-privacy regimes (ε

Förderung

  • National Key Research and Development Program of China

Zitieren

APA

Liu, Z., Han, Y., Song, Y., Zhang, Z., & Li, A. (2026). DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models. Electronics, 15(5), 1113. https://doi.org/10.3390/electronics15051113

Vancouver

Liu Z, Han Y, Song Y, Zhang Z, Li A. DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models. Electronics. 2026;15(5):1113. doi:10.3390/electronics15051113

BibTeX
@article{Liu2026,
  author = {Liu, Ziniu and Han, Yue and Song, Yang and Zhang, Zhuwei and Li, Aiping},
  title = {DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {5},
  pages = {1113},
  doi = {10.3390/electronics15051113}
}
Kostenlos starten →