EUVIMEDEUVIMED entdecken
Crossref

TeDri:Teacher-Driven Region Knowledge Distillation

Zherui Zhang, Changwei Wang, Rongtao Xu, Wenhao Xu, Xingtian Pei, Shibiao Xu, Wenbo Xu, Li Guo

IEEE Transactions on Knowledge and Data Engineering · 2026-06 · 38 (6) · 3975-3990

DOI: 10.1109/tkde.2026.3675551 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Knowledge and Data Engineering, 2026-06. Verfasst von Zherui Zhang und 7 weiteren Autor:innen. Die Arbeit nennt 55 Literaturangaben. Gefördert von: National Key Research and Development Program of China, Beijing Natural Science Foundation, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Knowledge distillation as a practical tool to enhance the performance of small-capacity student networks on downstream tasks comes at the cost of a lengthy distillation process due to the online inference of teacher networks, especially when there is a large capacity gap between them. Therefore, in this paper, we propose a fast distillation framework called TeDri based on region images by offline saving relevant regional information and its teacher guidance. Specifically, first, to alleviate the lack of diversity caused by the fixed augmentation path in region images, we propose Teacher-driven MixUp strategies with mild intensity and advocate binding the mixing factor $\lambda$ λ with teacher guidance confidence, where more confident category representations dominate the MixUp process. Furthermore, recognizing the need to evaluate these randomly cropped regions, and we propose region contrastive learning, encourage the student network to mimic the region partitioning behavior of the teacher, promoting a comprehensive understanding of global semantic content from multiple local perspectives. Finally, we introduce region mutual learning, employing spatial constraints among regions to require the student network towards consistent content interpretation across localized regions. Experiments on CIFAR-100 and ImageNet-1 K validate the effectiveness of the proposed TeDri, achieving competitive performance while significantly reducing training time.

Förderung

  • National Key Research and Development Program of China
  • Beijing Natural Science Foundation
  • National Natural Science Foundation of China
  • Taishan Scholars Program
  • Key R&D Program of Shandong Province, China
  • Shandong Provincial University Youth Innovation and Technology Support Program
  • Shandong Provincial Natural Science Foundation for Young Scholars Program
  • Qilu University of Technology

Zitieren

APA

Zhang, Z., Wang, C., Xu, R., Xu, W., Pei, X., Xu, S., Xu, W., & Guo, L. (2026). TeDri:Teacher-Driven Region Knowledge Distillation. IEEE Transactions on Knowledge and Data Engineering, 38(6), 3975-3990. https://doi.org/10.1109/tkde.2026.3675551

Vancouver

Zhang Z, Wang C, Xu R, Xu W, Pei X, Xu S, et al. TeDri:Teacher-Driven Region Knowledge Distillation. IEEE Transactions on Knowledge and Data Engineering. 2026;38(6):3975-3990. doi:10.1109/tkde.2026.3675551

BibTeX
@article{Zhang2026,
  author = {Zhang, Zherui and Wang, Changwei and Xu, Rongtao and Xu, Wenhao and Pei, Xingtian and Xu, Shibiao and Xu, Wenbo and Guo, Li},
  title = {TeDri:Teacher-Driven Region Knowledge Distillation},
  journal = {IEEE Transactions on Knowledge and Data Engineering},
  year = {2026},
  volume = {38},
  number = {6},
  pages = {3975--3990},
  doi = {10.1109/tkde.2026.3675551}
}
Kostenlos starten →