EUVIMEDEUVIMED entdecken
Crossref

DFMC: Feature-Driven Data-Free Knowledge Distillation

Zherui Zhang, Rongtao Xu, Changwei Wang, Wenhao Xu, Shunpeng Chen, Shibiao Xu, Guangyuan Xu, Li Guo

IEEE Transactions on Circuits and Systems for Video Technology · 2025-10 · 35 (10) · 9681-9694

DOI: 10.1109/tcsvt.2025.3565616 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-10. Verfasst von Zherui Zhang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 79 Literaturangaben. Gefördert von: Beijing Natural Science Foundation, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Data-Free Knowledge Distillation (DFKD) enables knowledge transfer from teacher networks without access to the real dataset. However, generator-based DFKD methods often suffer from insufficient diversity or low-confidence in synthetic images, negatively impacting student network performance. This paper introduces DFMC, a generative feature-driven framework to mitigate the inherent limitations of DFKD. We propose exploiting semantic description between generative feature domains to guide augmentation strategies, avoiding random abstract inputs caused by inconsistent semantic quality. Then, by applying noise to the generative features, we produce contrastive learning pairs indirectly, limiting the sampling range of the feature domain to encourage the student network to learn domain-invariant features. Finally, we guide the student network to deeply mimic the teacher’s layer-wise implicit classification behavior for the augmented synthetic images. Extensive experiments across various datasets and downstream tasks demonstrate the effectiveness of DFMC, achieving significant improvements while preventing student networks from overfitting to semantic ambiguous images.

Förderung

  • Beijing Natural Science Foundation
  • National Natural Science Foundation of China

Zitieren

APA

Zhang, Z., Xu, R., Wang, C., Xu, W., Chen, S., Xu, S., Xu, G., & Guo, L. (2025). DFMC: Feature-Driven Data-Free Knowledge Distillation. IEEE Transactions on Circuits and Systems for Video Technology, 35(10), 9681-9694. https://doi.org/10.1109/tcsvt.2025.3565616

Vancouver

Zhang Z, Xu R, Wang C, Xu W, Chen S, Xu S, et al. DFMC: Feature-Driven Data-Free Knowledge Distillation. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(10):9681-9694. doi:10.1109/tcsvt.2025.3565616

BibTeX
@article{Zhang2025,
  author = {Zhang, Zherui and Xu, Rongtao and Wang, Changwei and Xu, Wenhao and Chen, Shunpeng and Xu, Shibiao and Xu, Guangyuan and Guo, Li},
  title = {DFMC: Feature-Driven Data-Free Knowledge Distillation},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {10},
  pages = {9681--9694},
  doi = {10.1109/tcsvt.2025.3565616}
}
Kostenlos starten →