EUVIMEDEUVIMED entdecken
Crossref

An Efficient and Robust Cloud-Based Deep Learning With Knowledge Distillation

Zeyi Tao, Qi Xia, Songqing Cheng, Qun Li

IEEE Transactions on Cloud Computing · 01.04.2023 · 11 (2) · 1733-1745

DOI: 10.1109/tcc.2022.3160129 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Cloud Computing, erschienen am 01.04.2023. Verfasst von Zeyi Tao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 48 Literaturangaben. Gefördert von: National Science Foundation, Commonwealth Cyber Initiative.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, deep neural networks have shown extraordinary power in various practical learning tasks, especially in object detection, classification, natural language processing. However, deploying such large models on resource-constrained devices or embedded systems is challenging due to their high computational cost. Efforts such as model partition, pruning, or quantization have been used at the expense of accuracy loss. Knowledge distillation is a technique that transfers model knowledge from a well-trained model (teacher) to a smaller and shallow model (student). Instead of using a learning model on the cloud, we can deploy distilled models on various edge devices, significantly reducing the computational cost, memory usage and prolonging the battery lifetime. In this work, we propose a novel neuron manifold distillation (NMD) method, where the student models imitate the teacher's output distribution and learn the feature geometry of the teacher model. In addition, to further improve the cloud-based learning system reliability, we propose a confident prediction mechanism to calibrate the model predictions. We conduct experiments with different distillation configurations over multiple datasets. Our proposed method demonstrates a consistent improvement in accuracy-speed trade-offs for the distilled model.

Förderung

  • National Science Foundation
  • Commonwealth Cyber Initiative

Zitieren

APA

Tao, Z., Xia, Q., Cheng, S., & Li, Q. (2023). An Efficient and Robust Cloud-Based Deep Learning With Knowledge Distillation. IEEE Transactions on Cloud Computing, 11(2), 1733-1745. https://doi.org/10.1109/tcc.2022.3160129

Vancouver

Tao Z, Xia Q, Cheng S, Li Q. An Efficient and Robust Cloud-Based Deep Learning With Knowledge Distillation. IEEE Transactions on Cloud Computing. 2023;11(2):1733-1745. doi:10.1109/tcc.2022.3160129

BibTeX
@article{Tao2023,
  author = {Tao, Zeyi and Xia, Qi and Cheng, Songqing and Li, Qun},
  title = {An Efficient and Robust Cloud-Based Deep Learning With Knowledge Distillation},
  journal = {IEEE Transactions on Cloud Computing},
  year = {2023},
  volume = {11},
  number = {2},
  pages = {1733--1745},
  doi = {10.1109/tcc.2022.3160129}
}
Kostenlos starten →