Large Language Model Meets Graph Neural Network in Knowledge Distillation
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (16) · 17295-17304
DOI: 10.1609/aaai.v39i16.33901 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Shengxiang Hu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref).
Abstract (Originalsprache)
While Large Language Models (LLMs) show promise for Text-Attributed Graphs (TAGs) learning, their deployment is hindered by computational demands. Graph Neural Networks (GNNs) are efficient but struggle with TAGs' complex semantics. We propose LinguGKD, a novel LLM-to-GNN knowledge distillation framework that enables transferring both local semantic details and global structural information from LLMs to GNNs. First, it introduces TAG-oriented instruction tuning, enhancing LLMs with graph-specific knowledge through carefully designed prompts. Next, it develops a layer-adaptive multi-scale contrastive distillation strategy aligning LLM and GNN features at multiple granularities, from node-level to graph-level. Finally, the distilled GNNs combine the semantic richness of LLMs with the computational efficiency of traditional GNNs. Experiments demonstrate that LinguGKD outperforms existing graph distillation frameworks, the distilled simple GNNs achieve comparable or superior performance to more complex GNNs and teacher LLMs, while maintaining computational efficiency. This work bridges the gap between LLMs and GNNs, facilitating advanced graph learning in resource-constrained environments and providing a framework to leverage ongoing LLM advancements for GNN improvement.
Zitieren
APA
Hu, S., Zou, G., Yang, S., Lin, S., Gan, Y., Zhang, B., & Chen, Y. (2025). Large Language Model Meets Graph Neural Network in Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, 39(16), 17295-17304. https://doi.org/10.1609/aaai.v39i16.33901
Vancouver
Hu S, Zou G, Yang S, Lin S, Gan Y, Zhang B, et al. Large Language Model Meets Graph Neural Network in Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(16):17295-17304. doi:10.1609/aaai.v39i16.33901
BibTeX
@article{Hu2025,
author = {Hu, Shengxiang and Zou, Guobing and Yang, Song and Lin, Shiyi and Gan, Yanglan and Zhang, Bofeng and Chen, Yixin},
title = {Large Language Model Meets Graph Neural Network in Knowledge Distillation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {16},
pages = {17295--17304},
doi = {10.1609/aaai.v39i16.33901}
}
EUVIMED