Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge
ACM Transactions on Internet of Things · 30.11.2026 · 7 (4) · 1-30
DOI: 10.1145/3816431 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Internet of Things, online erschienen am 17.06.2026. Verfasst von Minh-Tri Nguyen und 2 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Multi-modality machine learning (ML) systems are increasingly deployed in edge environments, such as smart building and autonomous robotic applications. These systems enable analysis of complex subjects by performing ML inferences on multiple data sources provided by Internet of Things (IoT)-enabled devices. However, operating such ML systems for multi-tenant applications presents a runtime orchestration challenge to optimize the execution of multiple inference tasks across modalities. Analysis requests from different tenants often rely on different data sources with distinct quality requirements, resulting in a wide range of conflicting optimization objectives for runtime inference orchestration. The diversity of ML models and their performance variations further complicate the orchestration, particularly in scheduling inference tasks and distributing inference workloads on a heterogeneous edge system. This article addresses these challenges by introducing an adaptive orchestration that manages inference task scheduling for multi-tenant applications in a multi-modality ML system. Our orchestration supports a complex tradeoff between quality refinements and inference capability of inference services while scheduling multiple inference tasks under dynamic time constraints. The orchestration employs an efficient mechanism for selecting instances of inference services to distribute inference workloads across a heterogeneous edge cluster. This mechanism allows the orchestration to leverage cross-modal information to refine inference quality following the tenant objectives. Furthermore, our orchestration considers the performance variation caused by resource contentions, runtime failures, and explainability overheads while distributing workloads. Extensive experiments on two real-world edge applications in a common heterogeneous edge testbed demonstrate that our orchestration can improve inference accuracy by up to 10% and reduce the late response ratio by up to 18% compared to existing approaches within our multi-tenant edge scenarios.
Zitieren
APA
Nguyen, M., Nguyen, A., & Truong, H. (2026). Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge. ACM Transactions on Internet of Things, 7(4), 1-30. https://doi.org/10.1145/3816431
Vancouver
Nguyen M, Nguyen A, Truong H. Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge. ACM Transactions on Internet of Things. 2026;7(4):1-30. doi:10.1145/3816431
BibTeX
@article{Nguyen2026,
author = {Nguyen, Minh-Tri and Nguyen, Anh-Dung and Truong, Hong-Linh},
title = {Quality-driven Inference Orchestration for Multi-modality Machine Learning Systems at the Edge},
journal = {ACM Transactions on Internet of Things},
year = {2026},
volume = {7},
number = {4},
pages = {1--30},
doi = {10.1145/3816431}
}
EUVIMED