EUVIMEDEUVIMED entdecken
Crossref

An explainable AI framework for studying query complexity and hallucinations in LLM scheduling tasks

Baelyn Thantron, Gopal Gupta

Journal of Emerging Investigators · 2026

DOI: 10.59720/25-313 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Emerging Investigators, 2026. Verfasst von Baelyn Thantron und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) are AI systems that generate text and can assist with tasks such as answering questions and scheduling appointments. However, they are susceptible to “hallucinations,” where the model provides inaccurate information or misinterprets tool outputs. We hypothesized that increasing query difficulty (i.e., multi-day or ambiguous time window queries) would facilitate higher occurrences of tool-use hallucinations by LLMs in appointment scheduling tasks because models would misinterpret structured calendar data. To test our hypothesis, we built a framework combining GPT-4o with tool-use (LangChain), rule-based validation (Prolog), and explainability methods (LIME and attention visualizations) to test generated queries categorized as simple, complex, or edge case. For simple queries, there was a baseline hallucination rate of 10.2%, 14.8% for complex, and 18.5% for edge cases, with explainability analyses showing that increased error rates resulted from misplaced model attention to more difficult phrases. This framework not only helps diagnose query types that cause LLMs to hallucinate but also shows how we can mitigate these errors to make models more dependable in real-world settings.

Zitieren

APA

Thantron, B., & Gupta, G. (2026). An explainable AI framework for studying query complexity and hallucinations in LLM scheduling tasks. Journal of Emerging Investigators. https://doi.org/10.59720/25-313

Vancouver

Thantron B, Gupta G. An explainable AI framework for studying query complexity and hallucinations in LLM scheduling tasks. Journal of Emerging Investigators. 2026. doi:10.59720/25-313

BibTeX
@article{Thantron2026,
  author = {Thantron, Baelyn and Gupta, Gopal},
  title = {An explainable AI framework for studying query complexity and hallucinations in LLM scheduling tasks},
  journal = {Journal of Emerging Investigators},
  year = {2026},
  doi = {10.59720/25-313}
}
Kostenlos starten →