Metacognition and Cognitive Transparency for Trustworthy Human-Robot Teams
Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 377-386
DOI: 10.1609/aaaiss.v10i1.43429 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Sanjay Oruganti und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Effective human-robot teaming in safety-critical, risk-averse contexts benefits from more than tactical proficiency. It is supported by metacognition and cognitive transparency, which allow robotic teammates to recognize gaps in their own knowledge, assess whether their current understanding is sufficient to act, and expose their reasoning to human partners. We present an architectural approach to these capabilities and demonstrate them in a functioning system built on HARMONIC, a dual-layer architecture that integrates the OntoAgent cognitive architecture with reactive tactical control. We evaluate the system through a simulated collaborative search task involving two heterogeneous robots and a human operator. A controlled comparison with six large language models (LLMs) in the same strategic role, using identical tactical infrastructure and a prompt that did not enumerate plan preconditions, shows that metacognitive self-assessment, the most critical of these capabilities, was absent in 27 of 30 LLM trials, with models acting on incomplete information and hallucinating details rather than detecting knowledge gaps. High LLM task success often co-occurred with malformed or hallucinated action commands that were caught by the tactical layer's validation and fallback behaviors, showing both the risk of statistical planners and the value of architecturally enforced guardrails. These results suggest that metacognitive self-assessment and hallucination-free reasoning in collaborative human-robot teams are more readily supported by knowledge-grounded cognitive architectures, which provide them by design, than by language model scale alone.
Zitieren
APA
Oruganti, S., Nirenburg, S., McShane, M., & English, J. (2026). Metacognition and Cognitive Transparency for Trustworthy Human-Robot Teams. Proceedings of the AAAI Symposium Series, 10(1), 377-386. https://doi.org/10.1609/aaaiss.v10i1.43429
Vancouver
Oruganti S, Nirenburg S, McShane M, English J. Metacognition and Cognitive Transparency for Trustworthy Human-Robot Teams. Proceedings of the AAAI Symposium Series. 2026;10(1):377-386. doi:10.1609/aaaiss.v10i1.43429
BibTeX
@article{Oruganti2026,
author = {Oruganti, Sanjay and Nirenburg, Sergei and McShane, Marjorie and English, Jesse},
title = {Metacognition and Cognitive Transparency for Trustworthy Human-Robot Teams},
journal = {Proceedings of the AAAI Symposium Series},
year = {2026},
volume = {10},
number = {1},
pages = {377--386},
doi = {10.1609/aaaiss.v10i1.43429}
}
EUVIMED