EUVIMEDScopri EUVIMED
Open access · CC BY 4.0

DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models

Ziniu Liu, Yue Han, Yang Song, Zhuwei Zhang, Aiping Li

Electronics · 2026-03-07 · 15 (5) · 1113

DOI: 10.3390/electronics15051113 ↗

At a glance

Journal article in Electronics, published online on 2026-03-07. Written by Ziniu Liu and 4 co-authors. 34 references. The full text is freely available (open access, CC BY 4.0). Funded by: National Key Research and Development Program of China.

AbstractDati bibliograficiCiteRelated studies

Abstract (original language)

Steering large language models (LLMs) toward desired behaviors while preserving privacy is a critical challenge in AI alignment. Existing differentially private (DP) steering methods, such as PSA, add high-dimensional noise that can severely degrade steering accuracy. We propose DP-JL, a novel approach that combines Johnson–Lindenstrauss (JL) random projection with differential privacy to reduce noise while maintaining formal privacy guarantees. DP-JL projects steering vectors into a lower-dimensional space (dimension k) before adding DP noise, reducing total noise magnitude from O(d) to O(k) where k≪d, while the privacy budget ε remains unchanged. We evaluate DP-JL on seven behavioral datasets with LLaMA-2-7B, Mistral-7B, Qwen2.5-7B, and Gemma-2-9B, alongside general capability benchmarks (MMLU, TruthfulQA). All accuracy values are measured on held-out test sets. Results show that DP-JL achieves: (1) up to 22.76 percentage points higher steering accuracy than PSA on the myopic-reward dataset (at fixed privacy budget ε≈0.22, δ=10−5); (2) 91.7% win rate on sycophancy with an average accuracy improvement of 3.01 percentage points; (3) systematic advantages in high-privacy regimes (ε

Funding

  • National Key Research and Development Program of China

Cite

APA

Liu, Z., Han, Y., Song, Y., Zhang, Z., & Li, A. (2026). DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models. Electronics, 15(5), 1113. https://doi.org/10.3390/electronics15051113

Vancouver

Liu Z, Han Y, Song Y, Zhang Z, Li A. DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models. Electronics. 2026;15(5):1113. doi:10.3390/electronics15051113

BibTeX
@article{Liu2026,
  author = {Liu, Ziniu and Han, Yue and Song, Yang and Zhang, Zhuwei and Li, Aiping},
  title = {DP-JL: Differentially Private Steering via Johnson–Lindenstrauss Projection for Large Language Models},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {5},
  pages = {1113},
  doi = {10.3390/electronics15051113}
}
Start for free →