EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

AERIS: ADAPTIVE EXECUTOR-SCOPED RESOURCE MANAGEMENT FOR PRIVACY-PRESERVING DATA PROCESSING IN APACHE SPARK

YiNuo Wang, ChangHao Zhang

Eurasia Journal of Science and Technology · 2023

DOI: 10.61784/ejst4161 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Eurasia Journal of Science and Technology, 2023. Verfasst von YiNuo Wang und 1 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Privacy-preserving Spark pipelines introduce executor-local costs that conventional resource policies do not model: cryptographic context reconstruction, enlarged protected working sets, and encrypted spill. This paper presents AERIS, an adaptive executor-scoped controller that couples task memory grants with soft affinity to reusable privacy state. AERIS operates only on coarse metadata—partition size, operator class, key domain, and executor load—and therefore does not inspect plaintext, keys, or differentially private outputs. Its cost estimator is calibrated by real local microbenchmarks of AES-GCM, HMAC-SHA256, bounded Laplace-noised aggregation, scrypt context setup, and encrypted temporary-file spill. We evaluate the design with an event-driven prototype aligned with Spark stage, task, barrier, and executor semantics. The prototype uses the real Wisconsin Diagnostic Breast Cancer dataset and deterministic bootstrap expansion to generate balanced, skewed, and bursty traces. Across ten paired seeds, AERIS reduces makespan by 9.3%, 16.6%, and 10.4% relative to fixed uniform executor budgets. In the two memory-stressed workloads it reduces encrypted spill volume by 62.7% and 64.4%. Against a stronger reactive load-aware baseline, AERIS is 1.5% faster on balanced traces, 0.6% slower on skewed traces, and 5.5% slower on bursty traces, while reducing encrypted spill by 74.4% and 81.0% and lowering p95 service time by 1.0%–3.7%. Queue-inclusive p95 completion latency worsens under skew because longest-first dispatch and state reuse delay short tasks; this limitation is reported rather than hidden. The results support executor-scoped privacy state as a useful scheduling dimension, while also showing that spill avoidance and completion-tail objectives must be balanced explicitly. The artifact is a measurement-backed, trace-driven prototype, not a claim of measurements from a patched Spark cluster.

Zitieren

APA

Wang, Y., & Zhang, C. (2023). AERIS: ADAPTIVE EXECUTOR-SCOPED RESOURCE MANAGEMENT FOR PRIVACY-PRESERVING DATA PROCESSING IN APACHE SPARK. Eurasia Journal of Science and Technology. https://doi.org/10.61784/ejst4161

Vancouver

Wang Y, Zhang C. AERIS: ADAPTIVE EXECUTOR-SCOPED RESOURCE MANAGEMENT FOR PRIVACY-PRESERVING DATA PROCESSING IN APACHE SPARK. Eurasia Journal of Science and Technology. 2023. doi:10.61784/ejst4161

BibTeX
@article{Wang2023,
  author = {Wang, YiNuo and Zhang, ChangHao},
  title = {AERIS: ADAPTIVE EXECUTOR-SCOPED RESOURCE MANAGEMENT FOR PRIVACY-PRESERVING DATA PROCESSING IN APACHE SPARK},
  journal = {Eurasia Journal of Science and Technology},
  year = {2023},
  doi = {10.61784/ejst4161}
}
Kostenlos starten →