Event-Driven Multimodal Sensing and Computing for Context-Aware Home Monitoring Using Stereo Vision and Dietary Event Anchoring
Sensors · 28.07.2026 · 26 (15) · 4803
DOI: 10.3390/s26154803 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 28.07.2026. Verfasst von Zhaozhen Tong und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Engineering, Faculty of Engineering, Kobe University, 1-1 Rokkodai-cho, Nada, Kobe 657-8501, Japan. Hauptthemen laut MEDLINE: Diet. Die Arbeit nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Science and Technology Agency, Japan Society for the Promotion of Science, JSPS KAKENHI.
Abstract (Originalsprache)
Real-world home monitoring requires sensing systems that can capture daily behaviour without continuous raw-video retention or excessive user burden. However, domestic environments present irregular activity timing, fragmented human presence, asynchronous multimodal events, and privacy-sensitive data management. This study proposes an event-driven multimodal sensing and computing framework for context-aware home monitoring using stereo vision and dietary event anchoring. The framework integrates stereo RGB-based three-dimensional human motion sensing, dining-zone-triggered meal image acquisition, runtime event orchestration, timestamp-based cross-modal synchronization, privacy-aware local storage, and large-language-model-assisted dietary context interpretation. Instead of continuously recording all sensor streams, the system activates and organizes sensing through human presence detection, debounce logic, cooldown-based session control, and dining-zone occupancy events. Meal-related events are used as contextual anchors to associate motion sessions and dietary observations into synchronized behavioural episodes. The prototype was deployed for 11 consecutive days in a real kitchen–dining environment, with the stabilized real-time monitoring phase evaluated from 11 to 14 February 2026. During this phase, the system generated 26 event-driven motion sessions and 51,165 captured pose frames, of which 25,925 were valid. Sustained active sessions accounted for 30.8% of all sessions but contributed 81.5% of captured pose frames, indicating that event-driven orchestration concentrated motion data within behaviourally meaningful activity windows. Eight meal-related records were obtained, seven of which overlapped with motion sessions, resulting in 87.5% meal-event overlap coverage. Structured pose outputs required approximately 550 kB/min, corresponding to about 33 MB/h of recorded pose data. LLM-assisted meal-image interpretation achieved a mean absolute percentage error of 25.44%, supporting its use for coarse dietary-context description rather than precise nutritional quantification. However, this result is interpreted only as evidence for coarse dietary-context description and not as validation of a precise nutritional or clinical dietary assessment method. These results demonstrate the system-level feasibility of transforming irregular domestic observations into structured, temporally indexed, and privacy-aware multimodal behavioural records for future home monitoring applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Graduate School of Engineering, Faculty of Engineering, Kobe University, 1-1 Rokkodai-cho, Nada, Kobe 657-8501, Japan.
- Graduate School of Health Sciences, Kobe University, 7-10-2 Tomogaoka, Suma, Kobe 654-0142, Japan.
- Center of Mathematical and Data Sciences, Kobe University, 1-1 Rokkodai-cho, Nada, Kobe 657-8501, Japan.
ORCID-Profile: Zhaozhen Tong, Kumiko Ono, Sinan Chen
Förderung
- Japan Science and Technology Agency
- Japan Society for the Promotion of Science
- JSPS KAKENHI
- JST SICORP
Zitieren
APA
Tong, Z., Ono, K., Nakamura, M., & Chen, S. (2026). Event-Driven Multimodal Sensing and Computing for Context-Aware Home Monitoring Using Stereo Vision and Dietary Event Anchoring. Sensors, 26(15), 4803. https://doi.org/10.3390/s26154803
Vancouver
Tong Z, Ono K, Nakamura M, Chen S. Event-Driven Multimodal Sensing and Computing for Context-Aware Home Monitoring Using Stereo Vision and Dietary Event Anchoring. Sensors. 2026;26(15):4803. doi:10.3390/s26154803
BibTeX
@article{Tong2026,
author = {Tong, Zhaozhen and Ono, Kumiko and Nakamura, Masahide and Chen, Sinan},
title = {Event-Driven Multimodal Sensing and Computing for Context-Aware Home Monitoring Using Stereo Vision and Dietary Event Anchoring},
journal = {Sensors},
year = {2026},
volume = {26},
number = {15},
pages = {4803},
doi = {10.3390/s26154803}
}
EUVIMED