EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Intelligent Orchestration of Distributed Large Foundation Model Inference at the Edge

Fernando Koch, Aladin Djuhera, Alecio Binotto

Computer Networks and Communications · 05.09.2025 · 111-128

DOI: 10.37256/cnc.3220256807 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computer Networks and Communications, online erschienen am 05.09.2025. Verfasst von Fernando Koch und 2 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Foundation Models (LFMs), including multi-modal and generative models, promise to unlock new capabilities for next-generation Edge AI applications. However, performing inference with LFMs in resource-constrained and heterogeneous edge environments, such as Multi-access Edge Computing (MEC), presents significant challenges for workload orchestration due to time-varying network, compute, and storage conditions. In particular, current split inference strategies, which partition LFM layers across nodes, are not designed to adapt to fluctuating workloads, dynamic bandwidth conditions, or evolving privacy constraints in high-utilization MEC environments. In this work, we propose a novel adaptive split inference orchestration framework that elevates both the placement and partitioning of LFM layers to runtime-tunable variables. Specifically, our framework enables real-time, quality-of-service (QoS)-aware management of inference workloads by extending conventional orchestrators with three key services: (1) Capacity-aware workload distribution, which continuously profiles node resources and selects an optimal subset of MEC nodes; (2) Dynamic partition migration, which transparently relocates pre-cut LFM segments in response to changes in utilization or network conditions;(3) Real-time reconfiguration, which dynamically re-splits LFM layers to balance latency, throughput, and privacy. We formalize the joint placement-partitioning problem, outline a reference architecture and algorithmic workflow, and discuss applicability in representative smart city, V2X, and industrial edge scenarios.

Zitieren

APA

Koch, F., Djuhera, A., & Binotto, A. (2025). Intelligent Orchestration of Distributed Large Foundation Model Inference at the Edge. Computer Networks and Communications, 111-128. https://doi.org/10.37256/cnc.3220256807

Vancouver

Koch F, Djuhera A, Binotto A. Intelligent Orchestration of Distributed Large Foundation Model Inference at the Edge. Computer Networks and Communications. 2025:111-128. doi:10.37256/cnc.3220256807

BibTeX
@article{Koch2025,
  author = {Koch, Fernando and Djuhera, Aladin and Binotto, Alecio},
  title = {Intelligent Orchestration of Distributed Large Foundation Model Inference at the Edge},
  journal = {Computer Networks and Communications},
  year = {2025},
  pages = {111--128},
  doi = {10.37256/cnc.3220256807}
}
Kostenlos starten →