EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Prompt Engineering and Model Selection for LLM-Based Nutritional Estimation from Food Images: A Multi-Dataset Investigation

Shinichi Nakagawa iD, Akira Yamamoto iD

Nutrients · 21.06.2026 · 18 (12) · 2017

DOI: 10.3390/nu18122017 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Nutrients, online erschienen am 21.06.2026. Verfasst von Shinichi Nakagawa und 1 weiteren Autor:innen. Beteiligte Einrichtung: Research Institute of Info-Communication Medicine (RinCOM), Tokyo 184-0004, Japan. Hauptthemen laut MEDLINE: Large Language Models, Nutritive Value, Photography, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 14 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Objectives: Accurate estimation of nutritional content from food images has important applications in dietary assessment and public health surveillance. While large language models (LLMs) have shown promise for this task, the effects of prompt design and model selection on estimation accuracy remain poorly characterized.

Methods

We evaluated three Claude models (Haiku 4.5, Sonnet 4.6, Opus 4.6) for visual estimation of five mandatory nutritional components (energy, protein, fat, carbohydrate, and salt equivalent) across three datasets: NutriImage (691 Japanese meal photographs with dietitian-validated ground truth, after OCR-mask quality filtering), SNAPMe (1463 US meal photographs from a publicly available benchmark), and the Japan Branded Food Database (JBFD; 989–1000 packaged food product images). We systematically compared a default prompt and a visual estimation prompt explicitly instructing the model not to read any text or numbers visible in the image.

Results

The visual estimation prompt substantially improved accuracy when paired with a sufficiently capable model (energy R2: 0.23 for Haiku to 0.60 for Sonnet, JBFD). Sonnet and Opus substantially outperformed Haiku across all datasets, while differences between Sonnet and Opus were small (MedAPE difference 1–3 percentage points). Packaged food images (JBFD) yielded higher R2 than meal photographs. Salt equivalent showed consistently poor accuracy (MedAPE 34–64%). On SNAPMe, Sonnet achieved lower energy MAE (116.9 vs. 123.0 kcal, −4.9%) and lower MAE for protein (5.9 vs. 7.9 g, −25.7%) and fat (6.6 vs. 8.7 g, −24.5%) compared with a recent ChatGPT-5 study.

Conclusions

Claude Sonnet offers the best cost-performance balance for LLM-based nutritional estimation. Prompt design substantially affects accuracy, but only when paired with a sufficiently capable model; model visual recognition capability appears to be a key determinant of performance. These findings highlight the inherent difficulty of this task and provide practical guidance for dietary assessment system development.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsNutritive ValuePhotographyImage Processing, Computer-AssistedHumansDatabases, FactualJapan

Stichwörter der Autor:innen

ClaudeJBFDNutriImageSNAPMedietary assessmentfood imagelarge language modelnutritional estimationprompt engineering

Autor:innen und Institutionen

  • Research Institute of Info-Communication Medicine (RinCOM), Tokyo 184-0004, Japan.
  • Faculty of Health Data Science, Juntendo University, Tokyo 113-8421, Japan.

ORCID-Profile: Shinichi Nakagawa, Akira Yamamoto

Zitieren

APA

Nakagawa, S., & Yamamoto, A. (2026). Prompt Engineering and Model Selection for LLM-Based Nutritional Estimation from Food Images: A Multi-Dataset Investigation. Nutrients, 18(12), 2017. https://doi.org/10.3390/nu18122017

Vancouver

Nakagawa S, Yamamoto A. Prompt Engineering and Model Selection for LLM-Based Nutritional Estimation from Food Images: A Multi-Dataset Investigation. Nutrients. 2026;18(12):2017. doi:10.3390/nu18122017

BibTeX
@article{Nakagawa2026,
  author = {Nakagawa, Shinichi and Yamamoto, Akira},
  title = {Prompt Engineering and Model Selection for LLM-Based Nutritional Estimation from Food Images: A Multi-Dataset Investigation},
  journal = {Nutrients},
  year = {2026},
  volume = {18},
  number = {12},
  pages = {2017},
  doi = {10.3390/nu18122017}
}
Kostenlos starten →