Exploration of Stability Judgments: Assessing Multimodal LLMs in Game-Inspired Physical Reasoning Tasks
Applied Sciences · 21.10.2025 · 15 (20) · 11253
DOI: 10.3390/app152011253 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 21.10.2025. Verfasst von Mury Fajar Dewantoro und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This study extends our previous investigation into whether multimodal large language models (MLLMs) can reason about physical reasoning, using a game environment as the testbed. Stability served as a foundational scenario to probe model understanding of physical reasoning. We evaluated twelve models, combining those from the earlier study with six additional open-weight models, across three tasks designed to capture different aspects of reasoning. Human participants were included as a reference point, consistently achieving the highest accuracy, underscoring the gap between model and human performance. Among MLLMs, the GPT series continued to perform strongly, with GPT-4o showing reliable results in image-based tasks, while the Qwen2.5-VL series reached the highest overall scores in this extended study and in some cases surpassed commercial counterparts. Simpler binary tasks yielded balanced performance across modalities, suggesting that models can capture certain basic aspects of reasoning, whereas more complex multiple-choice tasks led to sharp declines in accuracy. Structured inputs such as XML improved results in the prediction task, where Qwen2.5-VL outperformed GPT variants in our earlier work. These findings demonstrate progress in scaling and modality design for physical reasoning, while reaffirming that human participants remain superior across all tasks.
Zitieren
APA
Dewantoro, M. F., Abdullah, F., Xia, Y., Khan, I., Thawonmas, R., Ouyang, W., & Bachtiar, F. A. (2025). Exploration of Stability Judgments: Assessing Multimodal LLMs in Game-Inspired Physical Reasoning Tasks. Applied Sciences, 15(20), 11253. https://doi.org/10.3390/app152011253
Vancouver
Dewantoro MF, Abdullah F, Xia Y, Khan I, Thawonmas R, Ouyang W, et al. Exploration of Stability Judgments: Assessing Multimodal LLMs in Game-Inspired Physical Reasoning Tasks. Applied Sciences. 2025;15(20):11253. doi:10.3390/app152011253
BibTeX
@article{Dewantoro2025,
author = {Dewantoro, Mury Fajar and Abdullah, Febri and Xia, Yi and Khan, Ibrahim and Thawonmas, Ruck and Ouyang, Wenwen and Bachtiar, Fitra Abdurrachman},
title = {Exploration of Stability Judgments: Assessing Multimodal LLMs in Game-Inspired Physical Reasoning Tasks},
journal = {Applied Sciences},
year = {2025},
volume = {15},
number = {20},
pages = {11253},
doi = {10.3390/app152011253}
}
EUVIMED