A Data-Quality-Aware Streaming Data Infrastructure for Continuous AI-Driven Risk and Decision Intelligence
Journal of Intelligent Decision Making and Information Science · 06.10.2026 · 3 (15s) · 1663-1683
DOI: 10.17762/jidmis.v3.4705 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Intelligent Decision Making and Information Science, online erschienen am 06.10.2026. Verfasst von Sriharsha Gudi. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Organizations are constantly relying on streaming data for assessing security risks and making AI-driven decisions to ensure data quality in this day and age. However, there are several problems that affect data quality, such as, duplicate, invalid, missing, out-of-order, and delayed events. These problems could affect accuracy and reliability of decisions through downstream processing. This study adopts the distributed and data-engineering-based architecture for data streaming by proposing a data-quality-aware streaming data pipeline, which combines schema verification, ingestion of heterogenous data, quarantine and repair systems, AI storage, and temporal processing. This study adopts a fraud-risk healthcare dataset with 1000 records as seed workload to transform into a managed stream. This study also introduces data degradation levels of 5%, 10%, and 20% across different failure conditions like invalid, duplicated, missing, out-of-order, delayed, and combined. The proposed framework is tested against batch and conventional streaming. When it comes to results, similar downstream performance was observed in Random Forest (RF) when there was clean data, with ROC-AUC score of 0.863 and F1 of 0.411. Interestingly, the quality-aware infrastructure showed improved performance in decision-stability at 20% degradation for out-of-order, missing, and combined conditions. For invalid data, DSI was raised from 0.937 under conventional pipeline to 0.977 under quality-aware pipeline. The computational overload was measurable in quality-aware pipeline, especially in throughput and memory usage when workloads were higher. Findings of the study raises reliability concern when data quality is combined into streaming environment directly. The proposed architecture provides the foundation to determine the trade-offs between overall performance, reliability, and assurance of data quality in AI-based decisions.
Zitieren
APA
Gudi, S. (2026). A Data-Quality-Aware Streaming Data Infrastructure for Continuous AI-Driven Risk and Decision Intelligence. Journal of Intelligent Decision Making and Information Science, 3(15s), 1663-1683. https://doi.org/10.17762/jidmis.v3.4705
Vancouver
Gudi S. A Data-Quality-Aware Streaming Data Infrastructure for Continuous AI-Driven Risk and Decision Intelligence. Journal of Intelligent Decision Making and Information Science. 2026;3(15s):1663-1683. doi:10.17762/jidmis.v3.4705
BibTeX
@article{Gudi2026,
author = {Gudi, Sriharsha},
title = {A Data-Quality-Aware Streaming Data Infrastructure for Continuous AI-Driven Risk and Decision Intelligence},
journal = {Journal of Intelligent Decision Making and Information Science},
year = {2026},
volume = {3},
number = {15s},
pages = {1663--1683},
doi = {10.17762/jidmis.v3.4705}
}
EUVIMED