Reconciling Streaming Semantics with Batch-Oriented ML Inference: Architectural Patterns for Production-Scale Multi-Modal Candidate Retrieval
East African Journal of Information Technology · 13.11.2026 · 9 (3) · 136-146
DOI: 10.37284/eajit.9.3.6275 ↗Kurz gefasst
Fachartikel in der Zeitschrift East African Journal of Information Technology, online erschienen am 13.11.2026. Verfasst von Sandeep Ayloo. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Production candidate-retrieval systems increasingly combine streaming ingestion with multi-modal embedding generation and vector-indexed search, yet the infrastructure components underlying each stage were not designed around a shared execution model. Stream processors such as Apache Flink favour record-at-a-time or micro-batched dataflow with strict checkpointing guarantees, while machine learning inference services achieve favourable throughput only when requests are batched, and vector databases such as Milvus assume comparatively infrequent mutation relative to read volume. This paper documents two recurring friction points at these pipeline boundaries: the granularity mismatch between per-record streaming calls and batch-efficient inference serving, and the write amplification created by high-churn item updates against approximate nearest neighbour indexes. Two architectural patterns addressing each friction point are presented: adaptive micro-batching with checkpoint-integrated buffering for the streaming-to-inference boundary, and lazy invalidation backed by an external version registry for the vector-mutation boundary. Their interaction is examined, along with the workload conditions under which each pattern's trade-offs are acceptable. The patterns are offered as a reusable analytical framework grounded in systems literature and practitioner experience, and their applicability depends on workload freshness and latency tolerances.
Zitieren
APA
Ayloo, S. (2026). Reconciling Streaming Semantics with Batch-Oriented ML Inference: Architectural Patterns for Production-Scale Multi-Modal Candidate Retrieval. East African Journal of Information Technology, 9(3), 136-146. https://doi.org/10.37284/eajit.9.3.6275
Vancouver
Ayloo S. Reconciling Streaming Semantics with Batch-Oriented ML Inference: Architectural Patterns for Production-Scale Multi-Modal Candidate Retrieval. East African Journal of Information Technology. 2026;9(3):136-146. doi:10.37284/eajit.9.3.6275
BibTeX
@article{Ayloo2026,
author = {Ayloo, Sandeep},
title = {Reconciling Streaming Semantics with Batch-Oriented ML Inference: Architectural Patterns for Production-Scale Multi-Modal Candidate Retrieval},
journal = {East African Journal of Information Technology},
year = {2026},
volume = {9},
number = {3},
pages = {136--146},
doi = {10.37284/eajit.9.3.6275}
}
EUVIMED