EUVIMEDEUVIMED entdecken
Crossref

Resource-Efficient Collaborative Edge Transformer Inference With Hybrid Model Parallelism

Shengyuan Ye, Bei Ouyang, Jiangsu Du, Liekang Zeng, Tianyi Qian, Wenzhong Ou, Xiaowen Chu, Deke Guo, Yutong Lu, Xu Chen

IEEE Transactions on Mobile Computing · 2025-10 · 24 (10) · 10945-10962

DOI: 10.1109/tmc.2025.3574695 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2025-10. Verfasst von Shengyuan Ye und 9 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: Guangdong S&T Programme, Basic and Applied Basic Research Foundation of Guangdong Province, Guangzhou Basic and Applied Basic Research Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformer-based models have unlocked a plethora of powerful intelligent applications at the edge, such as voice assistant in smart home. Traditional deployment approaches offload the inference workloads to the remote cloud server, which would induce substantial pressure on the backbone network as well as raise users’ privacy concerns. To address that, in-situ inference has been recently recognized for edge intelligence, but it still confronts significant challenges stemming from the conflict between intensive workloads and limited on-device computing resources. In this paper, we leverage our observation that many edge environments usually comprise a rich set of accompanying trusted edge devices with idle resources and propose Galaxy+ , a collaborative edge AI system that breaks the resource walls across heterogeneous edge devices for efficient Transformer inference acceleration. Galaxy+ introduces a novel hybrid model parallelism to orchestrate collaborative inference, along with a heterogeneity and memory-aware parallelism planning for fully exploiting the resource potential. To mitigate the impact of tensor synchronizations on inference latency under bandwidth-constrained edge environments, Galaxy+ devises a tile-based fine-grained overlapping of communication and computation. Furthermore, a fault-tolerant re-scheduling mechanism is developed to address device-level resource dynamics, ensuring stable and low-latency inference. Extensive evaluation based on prototype implementation demonstrates that Galaxy+ remarkably outperforms state-of-the-art approaches under various edge environment setups, achieving a $1.2\times$ 1 . 2 × to $4.24\times$ 4 . 24 × end-to-end latency reduction. Besides, Galaxy+ can adapt to device-level resource dynamics, swiftly rescheduling and restoring inference in the presence of unexpected straggler devices.

Förderung

  • Guangdong S&T Programme
  • Basic and Applied Basic Research Foundation of Guangdong Province
  • Guangzhou Basic and Applied Basic Research Program
  • Program for Guangdong Introducing Innovative and Entrepreneurial Teams
  • Guangzhou Municipal Joint Funding Project with Universities and Enterprises

Zitieren

APA

Ye, S., Ouyang, B., Du, J., Zeng, L., Qian, T., Ou, W., Chu, X., Guo, D., Lu, Y., & Chen, X. (2025). Resource-Efficient Collaborative Edge Transformer Inference With Hybrid Model Parallelism. IEEE Transactions on Mobile Computing, 24(10), 10945-10962. https://doi.org/10.1109/tmc.2025.3574695

Vancouver

Ye S, Ouyang B, Du J, Zeng L, Qian T, Ou W, et al. Resource-Efficient Collaborative Edge Transformer Inference With Hybrid Model Parallelism. IEEE Transactions on Mobile Computing. 2025;24(10):10945-10962. doi:10.1109/tmc.2025.3574695

BibTeX
@article{Ye2025,
  author = {Ye, Shengyuan and Ouyang, Bei and Du, Jiangsu and Zeng, Liekang and Qian, Tianyi and Ou, Wenzhong and Chu, Xiaowen and Guo, Deke and Lu, Yutong and Chen, Xu},
  title = {Resource-Efficient Collaborative Edge Transformer Inference With Hybrid Model Parallelism},
  journal = {IEEE Transactions on Mobile Computing},
  year = {2025},
  volume = {24},
  number = {10},
  pages = {10945--10962},
  doi = {10.1109/tmc.2025.3574695}
}
Kostenlos starten →