Building Trustworthy Data Pipelines for State Government Reporting and Decision Support
Frontiers in Computer Science and Artificial Intelligence · 20.12.2024 · 3 (2) · 89-105
DOI: 10.32996/fcsai.2024.3.2.11 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Computer Science and Artificial Intelligence, online erschienen am 20.12.2024. Verfasst von Catherine Ganduri. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
State agencies increasingly rely on automated data pipelines to assemble the figures that appear in statutory reports, performance dashboards, open data releases, and the briefing material that informs legislative and executive decisions. When such a pipeline fails quietly, the failure does not stay technical. A mistaken total, a silently dropped file, or a changed unit of measure can reach a decision maker with the full authority of an official publication. This paper examines how state government organizations can design, govern, and operate data pipelines whose outputs deserve the trust placed in them. The paper defines pipeline trustworthiness as the combination of data quality, traceability, protection, accountability, and operational reliability, and it presents a layered reference architecture in which staged validation gates, versioned rules, preserved raw data, and end-to-end lineage work alongside a defined stewardship model. It also proposes a five-level maturity model that agencies can use to assess current practice and to sequence improvements. The framework was developed through a design science approach grounded in the data quality, data governance, and data engineering literature and in the policy context that applies to state agencies, and it was evaluated analytically through a criteria-based comparison and a fault injection scenario walkthrough of a representative environmental reporting workflow. The walkthrough indicates that, for the six failure scenarios examined, gated validation, quarantine of failing records, and reproducible lineage would convert silent failure modes into visible and recoverable events, while clear stewardship keeps responsibility for quality with the people who understand the data. The paper concludes with practical recommendations for public sector data leaders and engineers, together with a discussion of limitations and directions for further research.
Zitieren
APA
Ganduri, C. (2024). Building Trustworthy Data Pipelines for State Government Reporting and Decision Support. Frontiers in Computer Science and Artificial Intelligence, 3(2), 89-105. https://doi.org/10.32996/fcsai.2024.3.2.11
Vancouver
Ganduri C. Building Trustworthy Data Pipelines for State Government Reporting and Decision Support. Frontiers in Computer Science and Artificial Intelligence. 2024;3(2):89-105. doi:10.32996/fcsai.2024.3.2.11
BibTeX
@article{Ganduri2024,
author = {Ganduri, Catherine},
title = {Building Trustworthy Data Pipelines for State Government Reporting and Decision Support},
journal = {Frontiers in Computer Science and Artificial Intelligence},
year = {2024},
volume = {3},
number = {2},
pages = {89--105},
doi = {10.32996/fcsai.2024.3.2.11}
}
EUVIMED