EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Real-Time Fraud ML on Spark Structured Streaming: Micro-Batch vs. Continuous Processing

Bhargav Vadgama

American Journal of Technology · 27.12.2025 · 4 (4) · 60-86

DOI: 10.58425/ajt.v4i4.465 ↗

Kurz gefasst

Fachartikel in der Zeitschrift American Journal of Technology, online erschienen am 27.12.2025. Verfasst von Bhargav Vadgama. Die Arbeit wurde bisher 2-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aim

The aim of this study is to evaluate the effectiveness of machine learning based real time fraud detection using Apache Spark Structured Streaming, with specific emphasis on comparing the Micro Batch and Continuous Processing modes.

Methods

The study develops and evaluates a real time fraud detection pipeline trained on the IEEE CIS Fraud Detection dataset. The methodology includes feature engineering, supervised machine learning models, and stream processing to support near real time fraud classification. Apache Spark Structured Streaming is implemented using both Micro Batch and Continuous Processing modes. Experimental evaluations compare the two modes across multiple performance metrics, including latency, precision, recall, resource utilization, and system reliability.

Results

The results show that the Micro Batch mode provides strong analytical capabilities, robust fault tolerance, and support for complex transformations, albeit with a slight increase in processing latency. In contrast, the Continuous Processing mode delivers significantly lower latency and higher throughput, making it suitable for environments requiring rapid fraud alerting. However, it demonstrates limitations in supported transformations and recovery mechanisms. The comparative analysis indicates that neither processing mode consistently outperforms the other across all evaluation criteria.

Conclusion

The study concludes that the choice between Micro Batch and Continuous Processing modes in Apache Spark Structured Streaming should be driven by specific application requirements rather than a general preference for one mode. Each mode presents distinct tradeoffs between latency, analytical flexibility, and reliability in real time fraud detection systems. Recommendations: The study recommends adopting a use case driven approach when selecting Spark processing modes for fraud detection applications. Future research should explore hybrid architectures that combine the strengths of both processing modes, as well as advanced techniques such as active learning and graph-based machine learning to enhance adaptability and accuracy in real time fraud mitigation systems.

Zitieren

APA

Vadgama, B. (2025). Real-Time Fraud ML on Spark Structured Streaming: Micro-Batch vs. Continuous Processing. American Journal of Technology, 4(4), 60-86. https://doi.org/10.58425/ajt.v4i4.465

Vancouver

Vadgama B. Real-Time Fraud ML on Spark Structured Streaming: Micro-Batch vs. Continuous Processing. American Journal of Technology. 2025;4(4):60-86. doi:10.58425/ajt.v4i4.465

BibTeX
@article{Vadgama2025,
  author = {Vadgama, Bhargav},
  title = {Real-Time Fraud ML on Spark Structured Streaming: Micro-Batch vs. Continuous Processing},
  journal = {American Journal of Technology},
  year = {2025},
  volume = {4},
  number = {4},
  pages = {60--86},
  doi = {10.58425/ajt.v4i4.465}
}
Kostenlos starten →