EUVIMEDEUVIMED entdecken
Crossref

AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization

Qiyang Li, Rui Kong, Yuchen Li, Hengyi Cai, Shuaiqiang Wang, Linghe Kong, Guihai Chen, Dawei Yin

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31680-31688

DOI: 10.1609/aaai.v40i37.40435 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Qiyang Li und 7 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The integration of dynamic, sparse structures like Mixture-of-Experts (MoE) with parameter-efficient adapters (e.g., LoRA) is a powerful technique for enhancing Large Language Models (LLMs). However, this architectural enhancement comes at a steep cost: despite minimal increases in computational load, the inference latency often skyrockets, leading to decoding speeds slowing by over 2.5 times. Through a fine-grained performance analysis, we pinpoint the primary bottleneck not in the computation itself, but in the severe overhead from fragmented, sequential CUDA kernel launches required for conventional dynamic routing. To address this challenge, we introduce AdaFuse, a framework built on a tight co-design between the algorithm and the underlying hardware system to enable efficient dynamic adapter execution. Departing from conventional layer-wise or block-wise routing, AdaFuse employs a token-level pre-gating strategy, which makes a single, global routing decision for all adapter layers before a token is processed. This ``decide-once, apply-everywhere'' approach effectively staticizes the execution path for each token, creating an opportunity for holistic optimization. We capitalize on this by developing a custom CUDA kernel that performs a fused switching operation, merging the parameters of all selected LoRA adapters into the backbone model in a single, efficient pass. Experimental results on popular open-source LLMs show that AdaFuse achieves accuracy on par with state-of-the-art dynamic adapters while drastically cutting decoding latency by a factor of over 2.4x, thereby bridging the gap between model capability and inference efficiency.

Zitieren

APA

Li, Q., Kong, R., Li, Y., Cai, H., Wang, S., Kong, L., Chen, G., & Yin, D. (2026). AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31680-31688. https://doi.org/10.1609/aaai.v40i37.40435

Vancouver

Li Q, Kong R, Li Y, Cai H, Wang S, Kong L, et al. AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31680-31688. doi:10.1609/aaai.v40i37.40435

BibTeX
@article{Li2026,
  author = {Li, Qiyang and Kong, Rui and Li, Yuchen and Cai, Hengyi and Wang, Shuaiqiang and Kong, Linghe and Chen, Guihai and Yin, Dawei},
  title = {AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {37},
  pages = {31680--31688},
  doi = {10.1609/aaai.v40i37.40435}
}
Kostenlos starten →