EUVIMEDEUVIMED entdecken
Crossref

C-GNN-PRUNE: A Unified Graph-Based Framework for Structure-Aware Pruning of Mixture-of-Experts Models

Lin Li, Yan Wang, Zhuopeng Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (27) · 22976-22984

DOI: 10.1609/aaai.v40i27.39462 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Lin Li und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The Mixture-of-Experts (MoE) architecture has emerged as a promising paradigm for scaling large language models (LLMs) by activating only a sparse subset of experts per input. However, its massive parameter size remains a major obstacle to efficient deployment. Existing pruning methods often ignore two key aspects: the intricate structural dependencies among experts and the heterogeneous importance of different layers. To tackle these issues, we propose C-GNN-PRUNE, a unified and structure-aware compression framework tailored for MoE models. Our method introduces an EntropyGuided Allocation Module that dynamically assigns pruning budgets by leveraging expert activation entropy, enabling adaptive handling of inter-layer heterogeneity. To preserve structural collaboration patterns, we construct an expert interaction graph that fuses functional similarity and routing behavior, and employ a GNN-Based Embedding Module to learn structure-aware expert representations. These embeddings, along with co-activation patterns, are fed into a Community Detection Module to identify expert clusters for structured pruning. Finally, an Activation-Aware Selection Module retains the most critical experts in each community, balancing sparsity and expressiveness. Experiments on multiple open-source MoE models demonstrate that C-GNN-PRUNE consistently outperforms prior methods under various pruning ratios, achieving better trade-offs between compression and accuracy. This framework provides a modular and effective solution for structure-preserving compression of large-scale MoE models.

Zitieren

APA

Li, L., Wang, Y., & Wang, Z. (2026). C-GNN-PRUNE: A Unified Graph-Based Framework for Structure-Aware Pruning of Mixture-of-Experts Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(27), 22976-22984. https://doi.org/10.1609/aaai.v40i27.39462

Vancouver

Li L, Wang Y, Wang Z. C-GNN-PRUNE: A Unified Graph-Based Framework for Structure-Aware Pruning of Mixture-of-Experts Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(27):22976-22984. doi:10.1609/aaai.v40i27.39462

BibTeX
@article{Li2026,
  author = {Li, Lin and Wang, Yan and Wang, Zhuopeng},
  title = {C-GNN-PRUNE: A Unified Graph-Based Framework for Structure-Aware Pruning of Mixture-of-Experts Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {27},
  pages = {22976--22984},
  doi = {10.1609/aaai.v40i27.39462}
}
Kostenlos starten →