EUVIMEDEUVIMED entdecken
Crossref

Large Language Model Meets Graph Neural Network in Knowledge Distillation

Shengxiang Hu, Guobing Zou, Song Yang, Shiyi Lin, Yanglan Gan, Bofeng Zhang, Yixin Chen

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (16) · 17295-17304

DOI: 10.1609/aaai.v39i16.33901 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Shengxiang Hu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

While Large Language Models (LLMs) show promise for Text-Attributed Graphs (TAGs) learning, their deployment is hindered by computational demands. Graph Neural Networks (GNNs) are efficient but struggle with TAGs' complex semantics. We propose LinguGKD, a novel LLM-to-GNN knowledge distillation framework that enables transferring both local semantic details and global structural information from LLMs to GNNs. First, it introduces TAG-oriented instruction tuning, enhancing LLMs with graph-specific knowledge through carefully designed prompts. Next, it develops a layer-adaptive multi-scale contrastive distillation strategy aligning LLM and GNN features at multiple granularities, from node-level to graph-level. Finally, the distilled GNNs combine the semantic richness of LLMs with the computational efficiency of traditional GNNs. Experiments demonstrate that LinguGKD outperforms existing graph distillation frameworks, the distilled simple GNNs achieve comparable or superior performance to more complex GNNs and teacher LLMs, while maintaining computational efficiency. This work bridges the gap between LLMs and GNNs, facilitating advanced graph learning in resource-constrained environments and providing a framework to leverage ongoing LLM advancements for GNN improvement.

Zitieren

APA

Hu, S., Zou, G., Yang, S., Lin, S., Gan, Y., Zhang, B., & Chen, Y. (2025). Large Language Model Meets Graph Neural Network in Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, 39(16), 17295-17304. https://doi.org/10.1609/aaai.v39i16.33901

Vancouver

Hu S, Zou G, Yang S, Lin S, Gan Y, Zhang B, et al. Large Language Model Meets Graph Neural Network in Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(16):17295-17304. doi:10.1609/aaai.v39i16.33901

BibTeX
@article{Hu2025,
  author = {Hu, Shengxiang and Zou, Guobing and Yang, Song and Lin, Shiyi and Gan, Yanglan and Zhang, Bofeng and Chen, Yixin},
  title = {Large Language Model Meets Graph Neural Network in Knowledge Distillation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {16},
  pages = {17295--17304},
  doi = {10.1609/aaai.v39i16.33901}
}
Kostenlos starten →