EUVIMEDEUVIMED entdecken
Crossref

Structured Knowledge Distillation for Dense Prediction

Yifan Liu, Changyong Shu, Jingdong Wang, Chunhua Shen

IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.06.2023 · 45 (6) · 7035-7049

DOI: 10.1109/tpami.2020.3001940 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.06.2023. Verfasst von Yifan Liu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 148-mal zitiert (Crossref) und nennt 74 Literaturangaben. Gefördert von: ARC DP.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this work, we consider transferring the structure information from large networks to compact ones for dense prediction tasks in computer vision. Previous knowledge distillation strategies used for dense prediction tasks often directly borrow the distillation scheme for image classification and perform knowledge distillation for each pixel separately, leading to sub-optimal performance. Here we propose to distill structured knowledge from large networks to compact networks, taking into account the fact that dense prediction is a structured prediction problem. Specifically, we study two structured distillation schemes: i) pair-wise distillation that distills the pair-wise similarities by building a static graph; and ii) holistic distillation that uses adversarial training to distill holistic knowledge. The effectiveness of our knowledge distillation approaches is demonstrated by experiments on three dense prediction tasks: semantic segmentation, depth estimation and object detection. Code is available at https://git.io/StructKD.

Förderung

  • ARC DP

Zitieren

APA

Liu, Y., Shu, C., Wang, J., & Shen, C. (2023). Structured Knowledge Distillation for Dense Prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6), 7035-7049. https://doi.org/10.1109/tpami.2020.3001940

Vancouver

Liu Y, Shu C, Wang J, Shen C. Structured Knowledge Distillation for Dense Prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(6):7035-7049. doi:10.1109/tpami.2020.3001940

BibTeX
@article{Liu2023,
  author = {Liu, Yifan and Shu, Changyong and Wang, Jingdong and Shen, Chunhua},
  title = {Structured Knowledge Distillation for Dense Prediction},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {6},
  pages = {7035--7049},
  doi = {10.1109/tpami.2020.3001940}
}
Kostenlos starten →