Benchmarking Problem-Solving AI Agents in a Dairy Farm Digital Twin Against Expert-Derived Reference Solutions
International Conference on AI Research · 18.11.2026 · 7 (1) · 348-359
DOI: 10.34190/icair.7.1.5699 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Conference on AI Research, online erschienen am 18.11.2026. Verfasst von Jan Saro und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
A digital twin may describe an animal accurately without helping a farmer decide what to do next. This study examines that gap through a blinded benchmark of problem-solving agents in a simulated dairy-farm environment. The benchmark comprises 500 scenarios across 13 classes, covering animal health, farm management, sensor problems and situations in which no intervention is needed. Decisions and the steps leading to them are scored against hidden, deterministic reference solutions written by the authors. Six systems were compared, including conventional decision-support systems (DSS) and three configurations of the same language model: one-shot, ReAct and a digital-twin (DT) agent with action simulation and environmental feedback. In the paired evaluation of 300 scenarios, the DT agent improved the correct action rate from 73.7% for the one-shot model to 85.7%, but did not significantly outperform ReAct on this endpoint. The machine-learning (ML) DSS remained strongest overall within the simulator distribution. The DT agent was comparatively effective in difficult and sensor-related scenarios, yet often intervened when the reference solution required no action. No unsafe decisions were observed for this agent under the benchmark rules, but its more elaborate workflow increased API use. These findings show why decision-support agents should be assessed not only on whether they identify a problem, but also on whether they gather useful evidence, select a proportionate response and recognise when to stop. The benchmark is a research environment, not a validated veterinary system.
Zitieren
APA
Saro, J., & Pavelka, M. (2026). Benchmarking Problem-Solving AI Agents in a Dairy Farm Digital Twin Against Expert-Derived Reference Solutions. International Conference on AI Research, 7(1), 348-359. https://doi.org/10.34190/icair.7.1.5699
Vancouver
Saro J, Pavelka M. Benchmarking Problem-Solving AI Agents in a Dairy Farm Digital Twin Against Expert-Derived Reference Solutions. International Conference on AI Research. 2026;7(1):348-359. doi:10.34190/icair.7.1.5699
BibTeX
@article{Saro2026,
author = {Saro, Jan and Pavelka, Miroslav},
title = {Benchmarking Problem-Solving AI Agents in a Dairy Farm Digital Twin Against Expert-Derived Reference Solutions},
journal = {International Conference on AI Research},
year = {2026},
volume = {7},
number = {1},
pages = {348--359},
doi = {10.34190/icair.7.1.5699}
}
EUVIMED