EUVIMEDEUVIMED entdecken
Crossref

A Cost-Aware Operator Migration Approach for Distributed Stream Processing System

Jiawei Tan, Zhuo Tang, Wentong Cai, Wen Jun Tan, Xiong Xiao, Jiapeng Zhang, Yi Gao, Kenli Li

IEEE Transactions on Cloud Computing · 2025-01 · 13 (1) · 441-454

DOI: 10.1109/tcc.2025.3538512 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Cloud Computing, 2025-01. Verfasst von Jiawei Tan und 7 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 41 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Science and Technology Program of Changsha.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Stream processing is integral to edge computing due to its low-latency attributes. Nevertheless, variability in user group sizes and disparate computing capabilities of edge devices necessitate frequent operator migrations within the stream. Moreover, intricate dependencies among stream operators often obscure the detection of potential bottleneck operators until an identified bottleneck is migrated in the stream. To address this, we propose a Cost-Aware Operator Migration (CAOM) scheme. The CAOM scheme incorporates a bottleneck operator detection mechanism that directly identifies all bottleneck operators based on task running metrics. This approach avoids multiple consecutive operator migrations in complex tasks, reducing the number of task interruptions caused by operator migration. Moreover, CAOM takes into account the temporal variance in operator migration costs. By factoring in the fluctuating data generation rate from data sources at different time intervals, CAOM selects the optimal start time for operator migration to minimize the amount of accumulated data during task interruptions. Finally, we implemented CAOM on Apache Flink and evaluated its performance using the WordCount and Nexmark applications. Our experiments show that CAOM effectively reduces the number of necessary operator migrations in tasks with complex topologies and decreases the latency overhead associated with operator migration compared to state-of-the-art schemes.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Science and Technology Program of Changsha
  • Natural Science Foundation
  • China Postdoctoral Science Foundation
  • Natural Science Foundation of Hunan Province

Zitieren

APA

Tan, J., Tang, Z., Cai, W., Tan, W. J., Xiao, X., Zhang, J., Gao, Y., & Li, K. (2025). A Cost-Aware Operator Migration Approach for Distributed Stream Processing System. IEEE Transactions on Cloud Computing, 13(1), 441-454. https://doi.org/10.1109/tcc.2025.3538512

Vancouver

Tan J, Tang Z, Cai W, Tan WJ, Xiao X, Zhang J, et al. A Cost-Aware Operator Migration Approach for Distributed Stream Processing System. IEEE Transactions on Cloud Computing. 2025;13(1):441-454. doi:10.1109/tcc.2025.3538512

BibTeX
@article{Tan2025,
  author = {Tan, Jiawei and Tang, Zhuo and Cai, Wentong and Tan, Wen Jun and Xiao, Xiong and Zhang, Jiapeng and Gao, Yi and Li, Kenli},
  title = {A Cost-Aware Operator Migration Approach for Distributed Stream Processing System},
  journal = {IEEE Transactions on Cloud Computing},
  year = {2025},
  volume = {13},
  number = {1},
  pages = {441--454},
  doi = {10.1109/tcc.2025.3538512}
}
Kostenlos starten →