EUVIMEDEUVIMED entdecken
Crossref

Semantic Lineage Pipelines for Automated Data Quality Remediation in Lakehouse Architectures

Girish Wali, Jessy Christadoss, Praveen Sivathapandi, Anita Kori, Chetan Bulla

AVE Trends in Intelligent Computing Systems · 2026

DOI: 10.64091/atics.2026.000284 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AVE Trends in Intelligent Computing Systems, 2026. Verfasst von Girish Wali und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

At a conceptual level, when examined carefully, traditional cleaning pipelines often operate in isolation, correcting errors based on static rules without understanding the upstream context or downstream impact., to some extent By embedding semantic understanding into the lineage tracking, the system can infer correct data values based on the relationships between entities rather than just column constraints., depending on contextual factors In many observed contexts, the rapid adoption of Lakehouse architectures has converged the flexibility of data lakes with the management capabilities of data warehouses., in several instances This study proposes a novel Semantic Lineage Pipeline, (as reflected in earlier discussions In many observed contexts, the study employed Apache Spark for processing, Delta Lake for storage, and a custom graph-based lineage parser. In a broader academic sense, researchers utilised a dataset comprising 491 instances of complex retail transaction logs and IoT sensor readings. The observed outcomes demonstrate that incorporating semantic lineage significantly improves the accuracy of automated repairs compared to traditional isolationist methods. to some extent (SLP) that leverages metadata graphs to automate data repair across multiple instances. From an interpretative angle, the proposed architecture reduces manual intervention time and increases the reliability of analytical dashboards derived from the Lakehouse.

Zitieren

APA

Wali, G., Christadoss, J., Sivathapandi, P., Kori, A., & Bulla, C. (2026). Semantic Lineage Pipelines for Automated Data Quality Remediation in Lakehouse Architectures. AVE Trends in Intelligent Computing Systems. https://doi.org/10.64091/atics.2026.000284

Vancouver

Wali G, Christadoss J, Sivathapandi P, Kori A, Bulla C. Semantic Lineage Pipelines for Automated Data Quality Remediation in Lakehouse Architectures. AVE Trends in Intelligent Computing Systems. 2026. doi:10.64091/atics.2026.000284

BibTeX
@article{Wali2026,
  author = {Wali, Girish and Christadoss, Jessy and Sivathapandi, Praveen and Kori, Anita and Bulla, Chetan},
  title = {Semantic Lineage Pipelines for Automated Data Quality Remediation in Lakehouse Architectures},
  journal = {AVE Trends in Intelligent Computing Systems},
  year = {2026},
  doi = {10.64091/atics.2026.000284}
}
Kostenlos starten →