A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans
Sensors · 2025
DOI: 10.3390/s25020477 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Erika Mori und 3 weiteren Autor:innen. Beteiligte Einrichtung: National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In recent years, advancements in the interaction and collaboration between humans and have garnered significant attention. Social intelligence plays a crucial role in facilitating natural interactions and seamless communication between humans and Artificial Intelligence (AI). To assess AI’s ability to understand human interactions and the components necessary for such comprehension, datasets like Social-IQ have been developed. However, these datasets often rely on a simplistic question-and-answer format and lack justifications for the provided answers. Furthermore, existing methods typically produce direct answers by selecting from predefined choices without generating intermediate outputs, which hampers interpretability and reliability. To address these limitations, we conducted a comprehensive evaluation of AI methods on a video-based Question Answering (QA) benchmark focused on human interactions, leveraging additional annotations related to human responses. Our analysis highlights significant differences between human and AI response patterns and underscores critical shortcomings in current benchmarks. We anticipate that these findings will guide the creation of more advanced datasets and represent an important step toward achieving natural communication between humans and AI.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan.
- Department of Electronics and Electrical Engineering, Faculty of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohoku-ku, Yokohama 223-8522, Japan.
ORCID-Profile: Erika Mori, Yue Qiu, Hirokatsu Kataoka, Yoshimitsu Aoki
Zitieren
APA
Mori, E., Qiu, Y., Kataoka, H., & Aoki, Y. (2025). A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans. Sensors. https://doi.org/10.3390/s25020477
Vancouver
Mori E, Qiu Y, Kataoka H, Aoki Y. A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans. Sensors. 2025. doi:10.3390/s25020477
BibTeX
@article{Mori2025,
author = {Mori, Erika and Qiu, Yue and Kataoka, Hirokatsu and Aoki, Yoshimitsu},
title = {A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans},
journal = {Sensors},
year = {2025},
doi = {10.3390/s25020477}
}
EUVIMED