DFMC: Feature-Driven Data-Free Knowledge Distillation
IEEE Transactions on Circuits and Systems for Video Technology · 2025-10 · 35 (10) · 9681-9694
DOI: 10.1109/tcsvt.2025.3565616 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-10. Verfasst von Zherui Zhang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 79 Literaturangaben. Gefördert von: Beijing Natural Science Foundation, National Natural Science Foundation of China.
Abstract (Originalsprache)
Data-Free Knowledge Distillation (DFKD) enables knowledge transfer from teacher networks without access to the real dataset. However, generator-based DFKD methods often suffer from insufficient diversity or low-confidence in synthetic images, negatively impacting student network performance. This paper introduces DFMC, a generative feature-driven framework to mitigate the inherent limitations of DFKD. We propose exploiting semantic description between generative feature domains to guide augmentation strategies, avoiding random abstract inputs caused by inconsistent semantic quality. Then, by applying noise to the generative features, we produce contrastive learning pairs indirectly, limiting the sampling range of the feature domain to encourage the student network to learn domain-invariant features. Finally, we guide the student network to deeply mimic the teacher’s layer-wise implicit classification behavior for the augmented synthetic images. Extensive experiments across various datasets and downstream tasks demonstrate the effectiveness of DFMC, achieving significant improvements while preventing student networks from overfitting to semantic ambiguous images.
Förderung
- Beijing Natural Science Foundation
- National Natural Science Foundation of China
Zitieren
APA
Zhang, Z., Xu, R., Wang, C., Xu, W., Chen, S., Xu, S., Xu, G., & Guo, L. (2025). DFMC: Feature-Driven Data-Free Knowledge Distillation. IEEE Transactions on Circuits and Systems for Video Technology, 35(10), 9681-9694. https://doi.org/10.1109/tcsvt.2025.3565616
Vancouver
Zhang Z, Xu R, Wang C, Xu W, Chen S, Xu S, et al. DFMC: Feature-Driven Data-Free Knowledge Distillation. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(10):9681-9694. doi:10.1109/tcsvt.2025.3565616
BibTeX
@article{Zhang2025,
author = {Zhang, Zherui and Xu, Rongtao and Wang, Changwei and Xu, Wenhao and Chen, Shunpeng and Xu, Shibiao and Xu, Guangyuan and Guo, Li},
title = {DFMC: Feature-Driven Data-Free Knowledge Distillation},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2025},
volume = {35},
number = {10},
pages = {9681--9694},
doi = {10.1109/tcsvt.2025.3565616}
}
EUVIMED