EUVIMEDEUVIMED entdecken
Crossref

Defending against knowledge distillation-based model extraction via feature covariance disalignment

Yao Zhang, Yang Li, Zhisong Pan

The Computer Journal · 15.11.2026 · 69 (11) · 1855-1864

DOI: 10.1093/comjnl/bxag063 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Computer Journal, online erschienen am 02.08.2026. Verfasst von Yao Zhang und 2 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Current research on knowledge distillation, especially data-free knowledge distillation, is getting deeper and deeper, which may lead to an increase in model stealing. To address this challenge, anti-distillation techniques have been developed. Anti-distillation aims to eliminate dark knowledge in teacher models in order to prevent potential thieves from illegally accessing the feature extraction capabilities of the models via knowledge distillation. Traditional anti-distillation methods apply perturbations in the output, but this indirectly affects the model feature layer and fails to completely remove the dark knowledge in the feature layer, resulting in limited effectiveness when counteracting feature distillation methods. To solve this problem, this paper innovatively introduces perturbations in the feature layer, utilizes the covariance matrix to describe the global relationship of the feature map, and extracts the subspace using eigenvalue to improve the training efficiency. Eventually, the dark knowledge in the anti-distillation model is effectively eliminated by the inverse alignment strategy in both scale and angle dimensions. Experimental results show that our method performs well on multiple datasets, such as CIFAR-100 and Tiny-ImageNet, and can effectively resist different state-of-the-art distillation methods including logits distillation, feature distillation, and data-free distillation, and achieves stronger anti-distillation capability and model performance.

Zitieren

APA

Zhang, Y., Li, Y., & Pan, Z. (2026). Defending against knowledge distillation-based model extraction via feature covariance disalignment. The Computer Journal, 69(11), 1855-1864. https://doi.org/10.1093/comjnl/bxag063

Vancouver

Zhang Y, Li Y, Pan Z. Defending against knowledge distillation-based model extraction via feature covariance disalignment. The Computer Journal. 2026;69(11):1855-1864. doi:10.1093/comjnl/bxag063

BibTeX
@article{Zhang2026,
  author = {Zhang, Yao and Li, Yang and Pan, Zhisong},
  title = {Defending against knowledge distillation-based model extraction via feature covariance disalignment},
  journal = {The Computer Journal},
  year = {2026},
  volume = {69},
  number = {11},
  pages = {1855--1864},
  doi = {10.1093/comjnl/bxag063}
}
Kostenlos starten →