EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Hierarchical Knowledge Distillation for Efficient Model Compression and Transfer: A Multi-Level Aggregation Approach

Titinunt Kitrungrotsakul, Preeyanuch Srichola

Information · 12.01.2026 · 17 (1) · 70

DOI: 10.3390/info17010070 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information, online erschienen am 12.01.2026. Verfasst von Titinunt Kitrungrotsakul und 1 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The success of large-scale deep learning models in remote sensing tasks has been transformative, enabling significant advances in image classification, object detection, and image–text retrieval. However, their computational and memory demands pose challenges for deployment in resource-constrained environments. Knowledge distillation (KD) alleviates these issues by transferring knowledge from a strong teacher to a student model, which can be compact for efficient deployment or architecturally matched to improve accuracy under the same inference budget. In this paper, we introduce Hierarchical Multi-Segment Knowledge Distillation (HIMS_KD), a multi-stage framework that sequentially distills knowledge from a teacher into multiple assistant models specialized in low-, mid-, and high-level representations, and then aggregates their knowledge into the final student. We integrate feature-level alignment, auxiliary similarity-logit alignment, and supervised loss during distillation. Experiments on benchmark remote sensing datasets (RSITMD and RSICD) show that HIMS_KD improves retrieval performance and enhances zero-shot classification; and when a compact student is used, it reduces deployment cost while retaining strong accuracy.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Kitrungrotsakul, T., & Srichola, P. (2026). Hierarchical Knowledge Distillation for Efficient Model Compression and Transfer: A Multi-Level Aggregation Approach. Information, 17(1), 70. https://doi.org/10.3390/info17010070

Vancouver

Kitrungrotsakul T, Srichola P. Hierarchical Knowledge Distillation for Efficient Model Compression and Transfer: A Multi-Level Aggregation Approach. Information. 2026;17(1):70. doi:10.3390/info17010070

BibTeX
@article{Kitrungrotsakul2026,
  author = {Kitrungrotsakul, Titinunt and Srichola, Preeyanuch},
  title = {Hierarchical Knowledge Distillation for Efficient Model Compression and Transfer: A Multi-Level Aggregation Approach},
  journal = {Information},
  year = {2026},
  volume = {17},
  number = {1},
  pages = {70},
  doi = {10.3390/info17010070}
}
Kostenlos starten →