Image-Based Nutritional Advisory System: Employing Multimodal Deep Learning for Food Classification and Nutritional Analysis
Applied Sciences · 28.04.2025 · 15 (9) · 4911
DOI: 10.3390/app15094911 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 28.04.2025. Verfasst von Sheng-Tzong Cheng und 2 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Accurate dietary assessment is essential for effective health management and disease prevention. However, conventional methods that rely on manual food logging and nutritional lookup are often time consuming and error prone. This study proposes an image-based nutritional advisory system that integrates multimodal deep learning to automate food classification, volume estimation, and dietary recommendation to address these limitations. The system employs a fine-tuned CLIP model for zero-shot food recognition, achieving high accuracy across diverse food categories, including unseen items. For volume measurement, a learning-based multi-view stereo (MVS) approach eliminates the need for specialized hardware, yielding reliable estimations with a mean absolute percentage error (MAPE) of 23.5% across standard food categories. Nutritional values are then calculated by referencing verified food composition databases. Furthermore, the system leverages a large language model (Llama 3) to generate personalized dietary advice tailored to individual health goals. The experimental results show that the system attains a top 1 classification accuracy of 91% on CNFOOD-241 and 80% on Food 101 and delivers high-quality recommendation texts with a BLEU-4 score of 45.13. These findings demonstrate the system’s potential as a practical and scalable tool for automated dietary management, offering improved precision, convenience, and user experience.
Zitieren
APA
Cheng, S., Lyu, Y., & Teng, C. (2025). Image-Based Nutritional Advisory System: Employing Multimodal Deep Learning for Food Classification and Nutritional Analysis. Applied Sciences, 15(9), 4911. https://doi.org/10.3390/app15094911
Vancouver
Cheng S, Lyu Y, Teng C. Image-Based Nutritional Advisory System: Employing Multimodal Deep Learning for Food Classification and Nutritional Analysis. Applied Sciences. 2025;15(9):4911. doi:10.3390/app15094911
BibTeX
@article{Cheng2025,
author = {Cheng, Sheng-Tzong and Lyu, Ya-Jin and Teng, Ching},
title = {Image-Based Nutritional Advisory System: Employing Multimodal Deep Learning for Food Classification and Nutritional Analysis},
journal = {Applied Sciences},
year = {2025},
volume = {15},
number = {9},
pages = {4911},
doi = {10.3390/app15094911}
}
EUVIMED