EUVIMEDEUVIMED entdecken
Crossref

Disaggregated State Management in Apache Flink® 2.0

Yuan Mei, Rui Xia, Zhaoqian Lan, Kaitian Hu, Lei Huang, Paris Carbone, Yanfei Lei, Vasiliki Kalavri, Han Yin, Feng Wang

Proceedings of the VLDB Endowment · 2025-08 · 18 (12) · 4846-4859

DOI: 10.14778/3750601.3750609 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 16.09.2025. Verfasst von Yuan Mei und 9 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 66 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present Apache Flink 2.0, an evolution of the popular stream processing system's architecture that decouples computation from state management. Flink 2.0 relies on a remote distributed file system (DFS) for primary state storage and uses local disks as a secondary cache, with state updates streamed continuously and directly to the DFS. To address the latency implications of remote storage, Flink 2.0 incorporates an asynchronous runtime execution model. Furthermore, Flink 2.0 introduces ForSt, a novel state store featuring a unified file system that enables faster and lightweight checkpointing, recovery, and reconfiguration with minimal intrusion to the existing Flink runtime architecture. Using a comprehensive set of Nexmark benchmarks and a large-scale stateful production workload, we evaluate Flink 2.0's large-state processing, checkpointing, and recovery mechanisms. Our results show significant performance improvements and reduced resource utilization compared to the baseline Flink 1.20 implementation. Specifically, we observe up to 94% reduction in checkpoint duration, up to 49× faster recovery after failures or a rescaling operation, and up to 50% cost savings.

Zitieren

APA

Mei, Y., Xia, R., Lan, Z., Hu, K., Huang, L., Carbone, P., Lei, Y., Kalavri, V., Yin, H., & Wang, F. (2025). Disaggregated State Management in Apache Flink® 2.0. Proceedings of the VLDB Endowment, 18(12), 4846-4859. https://doi.org/10.14778/3750601.3750609

Vancouver

Mei Y, Xia R, Lan Z, Hu K, Huang L, Carbone P, et al. Disaggregated State Management in Apache Flink® 2.0. Proceedings of the VLDB Endowment. 2025;18(12):4846-4859. doi:10.14778/3750601.3750609

BibTeX
@article{Mei2025,
  author = {Mei, Yuan and Xia, Rui and Lan, Zhaoqian and Hu, Kaitian and Huang, Lei and Carbone, Paris and Lei, Yanfei and Kalavri, Vasiliki and Yin, Han and Wang, Feng},
  title = {Disaggregated State Management in Apache Flink® 2.0},
  journal = {Proceedings of the VLDB Endowment},
  year = {2025},
  volume = {18},
  number = {12},
  pages = {4846--4859},
  doi = {10.14778/3750601.3750609}
}
Kostenlos starten →