EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge

Minh-Tri Nguyen, Anh-Dung Nguyen, Hong-Linh Truong

ACM Transactions on Internet of Things · 30.11.2026 · 7 (4) · 1-30

DOI: 10.1145/3816431 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Internet of Things, online erschienen am 17.06.2026. Verfasst von Minh-Tri Nguyen und 2 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modality machine learning (ML) systems are increasingly deployed in edge environments, such as smart building and autonomous robotic applications. These systems enable analysis of complex subjects by performing ML inferences on multiple data sources provided by Internet of Things (IoT)-enabled devices. However, operating such ML systems for multi-tenant applications presents a runtime orchestration challenge to optimize the execution of multiple inference tasks across modalities. Analysis requests from different tenants often rely on different data sources with distinct quality requirements, resulting in a wide range of conflicting optimization objectives for runtime inference orchestration. The diversity of ML models and their performance variations further complicate the orchestration, particularly in scheduling inference tasks and distributing inference workloads on a heterogeneous edge system. This article addresses these challenges by introducing an adaptive orchestration that manages inference task scheduling for multi-tenant applications in a multi-modality ML system. Our orchestration supports a complex tradeoff between quality refinements and inference capability of inference services while scheduling multiple inference tasks under dynamic time constraints. The orchestration employs an efficient mechanism for selecting instances of inference services to distribute inference workloads across a heterogeneous edge cluster. This mechanism allows the orchestration to leverage cross-modal information to refine inference quality following the tenant objectives. Furthermore, our orchestration considers the performance variation caused by resource contentions, runtime failures, and explainability overheads while distributing workloads. Extensive experiments on two real-world edge applications in a common heterogeneous edge testbed demonstrate that our orchestration can improve inference accuracy by up to 10% and reduce the late response ratio by up to 18% compared to existing approaches within our multi-tenant edge scenarios.

Zitieren

APA

Nguyen, M., Nguyen, A., & Truong, H. (2026). Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge. ACM Transactions on Internet of Things, 7(4), 1-30. https://doi.org/10.1145/3816431

Vancouver

Nguyen M, Nguyen A, Truong H. Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge. ACM Transactions on Internet of Things. 2026;7(4):1-30. doi:10.1145/3816431

BibTeX
@article{Nguyen2026,
  author = {Nguyen, Minh-Tri and Nguyen, Anh-Dung and Truong, Hong-Linh},
  title = {Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge},
  journal = {ACM Transactions on Internet of Things},
  year = {2026},
  volume = {7},
  number = {4},
  pages = {1--30},
  doi = {10.1145/3816431}
}
Kostenlos starten →