BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis
Journal of Machine Learning Advances · 16.11.2026 · 2 (1) · 36-45
DOI: 10.66879/jmla.2026.83 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Machine Learning Advances, online erschienen am 16.11.2026. Verfasst von Yuvjeeth Hosur Sreenivas und 2 weiteren Autor:innen.
Abstract (Originalsprache)
This paper presents BodySync AI, a multimodal calorie-tracking prototype that combines optical character recognition (OCR), object detection, pose estimation, and an agentic large language model (LLM). The system supports two tasks: extracting nutritional values from photographs of food-packaging labels, and analyzing body proportions from full-body images. Nutrition extraction uses a classical image-processing front end (perspective correction, denoising, illumination normalization, and contrast enhancement) followed by EasyOCR and rulebased parsing of calories, fat, carbohydrates, and protein. In parallel, we train a YOLOv8n nutrition-label detector on the Open Food Facts Nutrition Table Detection dataset and study the effect of training epochs, learning rate, optimizer, batch size, and input resolution in 14 controlled runs. The best configuration reaches an mAP@50 of 0.904 and an mAP@50-95 of 0.727 on the validation split. Body analysis uses MediaPipe Pose landmarks to compute normalized shoulder width, hip width, torso length, and shoulder-to-hip ratio, which feed a baseline rule-based bodytype estimate. A Groq-hosted Llama 3.1 8B agent exposes these components as tools through a natural-language interface. We describe the system as built, report the detector experiments, and outline the evaluations and learned body-composition models planned as future work.
Zitieren
APA
Sreenivas, Y. H., Mishra, A. R., & Silva, S. R. (2026). BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis. Journal of Machine Learning Advances, 2(1), 36-45. https://doi.org/10.66879/jmla.2026.83
Vancouver
Sreenivas YH, Mishra AR, Silva SR. BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis. Journal of Machine Learning Advances. 2026;2(1):36-45. doi:10.66879/jmla.2026.83
BibTeX
@article{Sreenivas2026,
author = {Sreenivas, Yuvjeeth Hosur and Mishra, Abhishek Raj and Silva, Stephanie Roselle},
title = {BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis},
journal = {Journal of Machine Learning Advances},
year = {2026},
volume = {2},
number = {1},
pages = {36--45},
doi = {10.66879/jmla.2026.83}
}
EUVIMED