Scalable Big Data Platform With End-to-End Traceability for Health Data Monitoring in Older Adults: Development and Performance Evaluation
JMIR Medical Informatics · 22.12.2025 · 13 · e81701-e81701
DOI: 10.2196/81701 ↗Kurz gefasst
Fachartikel in der Zeitschrift JMIR Medical Informatics, online erschienen am 22.12.2025. Verfasst von Ander Cejudo und 5 weiteren Autor:innen. Beteiligte Einrichtung: Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Mikeletegi 57, Donostia-San Sebastián, Basque Country, 20009, Spain, 34 943 309…. Hauptthemen laut MEDLINE: Big Data. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Themen: Gerontologische Pflege
Abstract (Originalsprache)
Background
The increasing use of real-time health data from wearable devices and self-reported questionnaires offers significant opportunities for preventive care in aging populations. However, current health data platforms often lack built-in mechanisms for data and model traceability, version control, and coordinated management of heterogeneous data streams, which are essential for clinical accountability, regulatory compliance, and reproducibility. The absence of these features limits the reuse of health data and the reproducibility of analytical workflows across research and clinical environments.
Objective
This work presents DeltaTrace, a unified big data health platform designed with traceability as a key architectural feature. The platform integrates end-to-end tracking of data and model versions with real-time and batch processing capabilities. Built entirely on open source technologies, DeltaTrace combines components for data management, model management, orchestration, and visualization. The main objective is to demonstrate that embedding traceability within the architecture enables scalable, auditable, and version-controlled processing of health data, thereby facilitating reproducible analytics and long-term maintenance of health monitoring systems.
Methods
DeltaTrace adopts a medallion architecture implemented with Delta Lake to ensure atomic and version-controlled data transformations. Apache Spark is used for distributed computation, Apache Kafka for continuous data ingestion, and Apache Airflow for orchestration of batch and streaming workflows. MLflow manages the lifecycle and versioning of machine learning models, while Grafana provides visualization dashboards for real-time and aggregated data inspection. The platform is evaluated using continuous physiological signals from wearable devices and batch-ingested questionnaire data, combining synthetic and real data from the LifeSnaps dataset. Performance tests are conducted on central processing unit–only servers with 8-core and 24-core configurations to assess ingestion, aggregation, visualization, and anomaly detection latency.
Results
DeltaTrace supports continuous processing for approximately 1500 users with end-to-end delays below 10 minutes. Ingestion and visualization tasks operate between mean 4.9 (SD 0.12) and 7.5 (SD 0.28) minutes, while aggregation and anomaly detection required less than mean 5.6 (SD 0.04) and 10.5 (SD 1.70) minutes, respectively. Increasing from 8 to 24 cores improved ingestion and cleaning latency by up to 25% and anomaly detection performance by up to 50%. The system maintains consistent performance across different data types, processing modes, and loads.
Conclusions
DeltaTrace provides a scalable and modular architecture that incorporates traceability as a core component together with functions for model management, orchestration, and visualization. The platform enables complete version control across data and models and maintains performance under limited hardware conditions. These characteristics support reproducible and auditable health data processing and make DeltaTrace suitable for continuous monitoring and preventive health care in aging populations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Vicomtech Foundation, Basque Research and Technology Alliance (BRTA), Mikeletegi 57, Donostia-San Sebastián, Basque Country, 20009, Spain, 34 943 309 230.
- Faculty of Engineering, University of Duesto, Bilbao, Spain.
- e-Health Department, Biodonostia Health Research Institute, San Sebastián, Spain.
- Computer Science and Artificial Intelligence Department, University of the Basque Country UPV/EHU, Donostia-San Sebastián, Spain.
ORCID-Profile: Ander Cejudo, Yone Tellechea, Amaia Calvo, Aitor Almeida, Cristina Martín, Andoni Beristain
Zitieren
APA
Cejudo, A., Tellechea, Y., Calvo, A., Almeida, A., Martín, C., & Beristain, A. (2025). Scalable Big Data Platform With End-to-End Traceability for Health Data Monitoring in Older Adults: Development and Performance Evaluation. JMIR Medical Informatics, 13, e81701-e81701. https://doi.org/10.2196/81701
Vancouver
Cejudo A, Tellechea Y, Calvo A, Almeida A, Martín C, Beristain A. Scalable Big Data Platform With End-to-End Traceability for Health Data Monitoring in Older Adults: Development and Performance Evaluation. JMIR Medical Informatics. 2025;13:e81701-e81701. doi:10.2196/81701
BibTeX
@article{Cejudo2025,
author = {Cejudo, Ander and Tellechea, Yone and Calvo, Amaia and Almeida, Aitor and Martín, Cristina and Beristain, Andoni},
title = {Scalable Big Data Platform With End-to-End Traceability for Health Data Monitoring in Older Adults: Development and Performance Evaluation},
journal = {JMIR Medical Informatics},
year = {2025},
volume = {13},
pages = {e81701--e81701},
doi = {10.2196/81701}
}
EUVIMED