A Lightweight Method for Graph Neural Networks Based on Knowledge Distillation and Graph Contrastive Learning
Applied Sciences · 02.06.2024 · 14 (11) · 4805
DOI: 10.3390/app14114805 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 02.06.2024. Verfasst von Yong Wang und 1 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key R&D Program of China.
Abstract (Originalsprache)
Graph neural networks (GNNs) are crucial tools for processing non-Euclidean data. However, due to scalability issues caused by the dependency and topology of graph data, deploying GNNs in practical applications is challenging. Some methods aim to address this issue by transferring GNN knowledge to MLPs through knowledge distillation. However, distilled MLPs cannot directly capture graph structure information and rely only on node features, resulting in poor performance and sensitivity to noise. To solve this problem, we propose a lightweight optimization method for GNNs that combines graph contrastive learning and variable-temperature knowledge distillation. First, we use graph contrastive learning to capture graph structural representations, enriching the input information for the MLP. Then, we transfer GNN knowledge to the MLP using variable temperature knowledge distillation. Additionally, we enhance both node content and structural features before inputting them into the MLP, thus improving its performance and stability. Extensive experiments on seven datasets show that the proposed KDGCL model outperforms baseline models in both transductive and inductive settings; in particular, the KDGCL model achieves an average improvement of 1.63% in transductive settings and 0.8% in inductive settings when compared to baseline models. Furthermore, KDGCL maintains parameter efficiency and inference speed, making it competitive in terms of performance.
Förderung
- National Key R&D Program of China
Zitieren
APA
Wang, Y., & Yang, S. (2024). A Lightweight Method for Graph Neural Networks Based on Knowledge Distillation and Graph Contrastive Learning. Applied Sciences, 14(11), 4805. https://doi.org/10.3390/app14114805
Vancouver
Wang Y, Yang S. A Lightweight Method for Graph Neural Networks Based on Knowledge Distillation and Graph Contrastive Learning. Applied Sciences. 2024;14(11):4805. doi:10.3390/app14114805
BibTeX
@article{Wang2024,
author = {Wang, Yong and Yang, Shuqun},
title = {A Lightweight Method for Graph Neural Networks Based on Knowledge Distillation and Graph Contrastive Learning},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {11},
pages = {4805},
doi = {10.3390/app14114805}
}
EUVIMED