EUVIMEDEUVIMED entdecken
Crossref

OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance from Database Query Event Logs

Fotis Psallidas, Ashvin Agrawal, Chandru Sugunan, Khaled Ibrahim, Konstantinos Karanasos, Jesús Camacho-Rodríguez, Avrilia Floratou, Carlo Curino, Raghu Ramakrishnan

Proceedings of the VLDB Endowment · 2023-08 · 16 (12) · 3662-3675

DOI: 10.14778/3611540.3611555 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, 2023-08. Verfasst von Fotis Psallidas und 8 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 86 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Provenance encodes information that connects datasets, their generation workflows, and associated metadata (e.g., who or when executed a query). As such, it is instrumental for a wide range of critical governance applications (e.g., observability and auditing). Unfortunately, in the context of database systems, extracting coarse-grained provenance is a long-standing problem due to the complexity and sheer volume of database workflows. Provenance extraction from query event logs has been recently proposed as favorable because, in principle, can result in meaningful provenance graphs for provenance applications. Current approaches, however, (a) add substantial overhead to the database and provenance extraction workflows and (b) extract provenance that is noisy, omits query execution dependencies, and is not rich enough for upstream applications. To address these problems, we introduce OneProvenance: an efficient provenance extraction system from query event logs. OneProvenance addresses the unique challenges of log-based extraction by (a) identifying query execution dependencies through efficient log analysis, (b) extracting provenance through novel event transformations that account for query dependencies, and (c) introducing effective filtering optimizations. Our thorough experimental analysis shows that OneProvenance can improve extraction by up to ~18X compared to state-of-the-art baselines; our optimizations reduce the extraction noise and optimize performance even further. OneProvenance is deployed at scale by Microsoft Purview and actively supports customer provenance extraction needs (https://bit.ly/3N2JVGF).

Zitieren

APA

Psallidas, F., Agrawal, A., Sugunan, C., Ibrahim, K., Karanasos, K., Camacho-Rodríguez, J., Floratou, A., Curino, C., & Ramakrishnan, R. (2023). OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance from Database Query Event Logs. Proceedings of the VLDB Endowment, 16(12), 3662-3675. https://doi.org/10.14778/3611540.3611555

Vancouver

Psallidas F, Agrawal A, Sugunan C, Ibrahim K, Karanasos K, Camacho-Rodríguez J, et al. OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance from Database Query Event Logs. Proceedings of the VLDB Endowment. 2023;16(12):3662-3675. doi:10.14778/3611540.3611555

BibTeX
@article{Psallidas2023,
  author = {Psallidas, Fotis and Agrawal, Ashvin and Sugunan, Chandru and Ibrahim, Khaled and Karanasos, Konstantinos and Camacho-Rodríguez, Jesús and Floratou, Avrilia and Curino, Carlo and Ramakrishnan, Raghu},
  title = {OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance from Database Query Event Logs},
  journal = {Proceedings of the VLDB Endowment},
  year = {2023},
  volume = {16},
  number = {12},
  pages = {3662--3675},
  doi = {10.14778/3611540.3611555}
}
Kostenlos starten →