EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Real-Time Big Data Engineering: A Distributed Processing Framework for Cloud-Based Streaming Analytics and Operational Intelligence

Sai Prakhyath Jawaji

Journal of Intelligent Decision Making and Information Science · 15.09.2026 · 3 (19s) · 1531-1562

DOI: 10.17762/jidmis.v3.6343 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Intelligent Decision Making and Information Science, online erschienen am 15.09.2026. Verfasst von Sai Prakhyath Jawaji. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The increasing dependence of enterprises on continuously generated data has transformed real-time processing from a specialized technical capability into a fundamental requirement of modern operational intelligence. Telemetry streams, transactional events, application logs, connected devices, digital interactions, and infrastructure signals increasingly need to be interpreted while they are still operationally relevant. Yet reducing latency alone does not produce reliable real-time intelligence. Enterprise streaming architectures face a fundamental tension between the necessity for low-latency, real-time data ingestion and the operational requirement for stateful, highly accurate processing that guarantees data integrity. This study develops a distributed processing framework for resolving this tension without treating speed and correctness as mutually exclusive architectural objectives. Rather than reproducing a conventional three-layer model, the proposed framework conceptualizes real-time big data engineering as a continuous event-to-intelligence lifecycle in which events are durably captured, temporally interpreted, statefully computed, reconciled with historical information, and progressively transformed into operationally actionable intelligence. Particular attention is given to event-time semantics, late and out-of-order data, distributed state management, incremental computation, fault recovery, backpressure, stream–history integration, and consistency across real-time and retrospective analytical views. The framework further examines how workload partitioning, state locality, checkpointing, replayability, and adaptive processing can preserve low latency as data velocity and computational complexity increase. The resulting architecture demonstrates that enterprise streaming systems should not be engineered merely to process events faster, but to preserve their meaning and computational correctness while reducing the interval between observation and informed action. In this model, real-time big data engineering becomes the technical foundation for operational intelligence systems capable of continuously sensing, interpreting, and responding to changing enterprise conditions.

Zitieren

APA

Jawaji, S. P. (2026). Real-Time Big Data Engineering: A Distributed Processing Framework for Cloud-Based Streaming Analytics and Operational Intelligence. Journal of Intelligent Decision Making and Information Science, 3(19s), 1531-1562. https://doi.org/10.17762/jidmis.v3.6343

Vancouver

Jawaji SP. Real-Time Big Data Engineering: A Distributed Processing Framework for Cloud-Based Streaming Analytics and Operational Intelligence. Journal of Intelligent Decision Making and Information Science. 2026;3(19s):1531-1562. doi:10.17762/jidmis.v3.6343

BibTeX
@article{Jawaji2026,
  author = {Jawaji, Sai Prakhyath},
  title = {Real-Time Big Data Engineering: A Distributed Processing Framework for Cloud-Based Streaming Analytics and Operational Intelligence},
  journal = {Journal of Intelligent Decision Making and Information Science},
  year = {2026},
  volume = {3},
  number = {19s},
  pages = {1531--1562},
  doi = {10.17762/jidmis.v3.6343}
}
Kostenlos starten →