Methodology for Evaluating Artificial Intelligence Chatbots: A Review with Applications to Agricultural Advisory Systems
Journal of Sustainable Technology in Agriculture · 16.11.2026 · 2 (4)
DOI: 10.65287/josta.202610.de78 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Sustainable Technology in Agriculture, online erschienen am 16.11.2026. Verfasst von Farhana L S.
Abstract (Originalsprache)
Artificial intelligence chatbots increasingly advise farmers, yet their adoption should rest on rigorous, standardised evaluation. This review brings together five standardised instruments for assessing chatbots (the System Usability Scale, the Chatbot Usability Scale, the Checklist for Trust between People and Automation, the AI Chatbots Usage Scale, and the User Engagement Scale) and one assessment method, knowledge testing. It also explains Cronbach’s alpha, the reliability criterion most of them rely on. A case study from the literature, in which five large language models answered PhD-level soil science questions, together with deployed systems such as Farmer.Chat and AgroLLM, show how evaluation works in practice and why fluent but unverified advice is risky. Beyond questionnaires, we examine accuracy-based and retrieval-grounded testing. We propose a five-layer protocol that first checks technical accuracy, then usability, trust, engagement, and learning outcomes, to guide the design and validation of chatbots for precision and smallholder farming.
Zitieren
APA
S, F. L. (2026). Methodology for Evaluating Artificial Intelligence Chatbots: A Review with Applications to Agricultural Advisory Systems. Journal of Sustainable Technology in Agriculture, 2(4). https://doi.org/10.65287/josta.202610.de78
Vancouver
S FL. Methodology for Evaluating Artificial Intelligence Chatbots: A Review with Applications to Agricultural Advisory Systems. Journal of Sustainable Technology in Agriculture. 2026;2(4). doi:10.65287/josta.202610.de78
BibTeX
@article{S2026,
author = {S, Farhana L},
title = {Methodology for Evaluating Artificial Intelligence Chatbots: A Review with Applications to Agricultural Advisory Systems},
journal = {Journal of Sustainable Technology in Agriculture},
year = {2026},
volume = {2},
number = {4},
doi = {10.65287/josta.202610.de78}
}
EUVIMED