POPKI: Modeling Surprise in a Physically-Grounded Joint Inference of Observation, Preference, and Knowledge
Open Mind · 17.07.2026 · 10 · 1157-1170
DOI: 10.1162/opmi.a.375 ↗Kurz gefasst
Fachartikel in der Zeitschrift Open Mind, online erschienen am 17.07.2026. Verfasst von Harry Chen und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: DARPA Machine Common Sense, MIT-IBM AI LAB, NSF STC.
Abstract (Originalsprache)
People naturally take into account others’ limited perceptual access when attributing mental states such as preferences and knowledge. This ability is central to intuitive psychology, and even infants recognize when others have only partial observability. Understanding how mental state attributions adapt under limited perceptual access is thus crucial for advancing models of intuitive psychology. Here, we introduce POPKI (Physically-grounded Observation, Preference, and Knowledge Inference), a Bayesian inverse-planning framework designed to model graded surprise based on agents’ inferred preferences and knowledge in visually constrained 3D environments. Building on and expanding the AGENT dataset, we develop new scenarios that test how partial observability shapes judgments of surprise. Despite the tremendous progress of state-of-the-art vision-language models (VLMs), our experiments show that they fail to capture the nuances of perceptual access when judging surprise. POPKI, in contrast, offers a principled approach to modeling human-like uncertainty under partial observability and closely replicates human surprise patterns. It provides a foundation for advancing psychological reasoning models and enhancing human-AI interaction.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
- Department of Electrical Engineering & Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
- MIT-IBM Watson AI Lab, Massachusetts Institute of Technology, Cambridge, MA, USA.
- Department of Psychology, Harvard University, Cambridge, MA, USA.
- Department of Computer Science & Department of Cognitive Science, Johns Hopkins University, Baltimore, MD, USA.
ORCID-Profile: Harry Chen, Karen Chung, Joshua B Tenenbaum
Förderung
- DARPA Machine Common Sense
- MIT-IBM AI LAB
- NSF STC
Zitieren
APA
Chen, H., Chung, K., Bhandwaldar, A., Tenenbaum, J. B., Ullman, T. D., & Shu, T. (2026). POPKI: Modeling Surprise in a Physically-Grounded Joint Inference of Observation, Preference, and Knowledge. Open Mind, 10, 1157-1170. https://doi.org/10.1162/opmi.a.375
Vancouver
Chen H, Chung K, Bhandwaldar A, Tenenbaum JB, Ullman TD, Shu T. POPKI: Modeling Surprise in a Physically-Grounded Joint Inference of Observation, Preference, and Knowledge. Open Mind. 2026;10:1157-1170. doi:10.1162/opmi.a.375
BibTeX
@article{Chen2026,
author = {Chen, Harry and Chung, Karen and Bhandwaldar, Abhishek and Tenenbaum, Joshua B and Ullman, Tomer D and Shu, Tianmin},
title = {POPKI: Modeling Surprise in a Physically-Grounded Joint Inference of Observation, Preference, and Knowledge},
journal = {Open Mind},
year = {2026},
volume = {10},
pages = {1157--1170},
doi = {10.1162/opmi.a.375}
}
EUVIMED