EUVIMEDEUVIMED entdecken
Crossref

Adaptive hierarchical knowledge distillation from GNNs to MLPs

Junfeng Zhang, Cheng Xie, BeiBei Yu, Rui Yang

Knowledge and Information Systems · 2025-09 · 67 (9) · 7619-7639

DOI: 10.1007/s10115-025-02447-w ↗

Kurz gefasst

Fachartikel in der Zeitschrift Knowledge and Information Systems, online erschienen am 21.05.2025. Verfasst von Junfeng Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 34 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Knowledge distillation has attracted much attention in knowledge and information systems. The latest distillation methods use Kullback–Leibler divergence to distill complex models from simple models, yielding surprising results. However, due to the difference in the capacity of graph neural networks (GNNs) and MLPs models, distilling GNNs through fixed dimensions and layers of MLPs will inevitably cause information loss. In addition, the gap between the features produced by GNNs and the features produced by MLPs gradually becomes larger as the number of layers increases. To this end, we propose an adaptive hierarchical distillation framework that distills GNNs through MLPs with variable dimensions and layers to ensure the integrity of the distilled information. Specifically, we use a neural architecture search to adaptively find MLPs with appropriate dimensions and layers for each layer of GNNs. Then, the graph structure information is distilled from GNNs layer by layer, which makes the student neural network structure can better match with the teacher model, and the features are better aligned, so as to better learn the graph structure information. At last, the student model distilled to the complete information is used in the downstream learning task. Experimental results on various datasets show impressive improvements in node classification tasks compared with previous state-of-the-art methods.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Zhang, J., Xie, C., Yu, B., & Yang, R. (2025). Adaptive hierarchical knowledge distillation from GNNs to MLPs. Knowledge and Information Systems, 67(9), 7619-7639. https://doi.org/10.1007/s10115-025-02447-w

Vancouver

Zhang J, Xie C, Yu B, Yang R. Adaptive hierarchical knowledge distillation from GNNs to MLPs. Knowledge and Information Systems. 2025;67(9):7619-7639. doi:10.1007/s10115-025-02447-w

BibTeX
@article{Zhang2025,
  author = {Zhang, Junfeng and Xie, Cheng and Yu, BeiBei and Yang, Rui},
  title = {Adaptive hierarchical knowledge distillation from GNNs to MLPs},
  journal = {Knowledge and Information Systems},
  year = {2025},
  volume = {67},
  number = {9},
  pages = {7619--7639},
  doi = {10.1007/s10115-025-02447-w}
}
Kostenlos starten →