EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans

Erika Mori iD, Yue Qiu iD, Hirokatsu Kataoka iD, Yoshimitsu Aoki iD

Sensors · 2025

DOI: 10.3390/s25020477 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Erika Mori und 3 weiteren Autor:innen. Beteiligte Einrichtung: National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, advancements in the interaction and collaboration between humans and have garnered significant attention. Social intelligence plays a crucial role in facilitating natural interactions and seamless communication between humans and Artificial Intelligence (AI). To assess AI’s ability to understand human interactions and the components necessary for such comprehension, datasets like Social-IQ have been developed. However, these datasets often rely on a simplistic question-and-answer format and lack justifications for the provided answers. Furthermore, existing methods typically produce direct answers by selecting from predefined choices without generating intermediate outputs, which hampers interpretability and reliability. To address these limitations, we conducted a comprehensive evaluation of AI methods on a video-based Question Answering (QA) benchmark focused on human interactions, leveraging additional annotations related to human responses. Our analysis highlights significant differences between human and AI response patterns and underscores critical shortcomings in current benchmarks. We anticipate that these findings will guide the creation of more advanced datasets and represent an important step toward achieving natural communication between humans and AI.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceHumansCommunicationLarge Language Models

Stichwörter der Autor:innen

VideoQAemotion recognitionhuman–robot interactionlarge language models (LLMs)social intelligenceunderstanding human behavior

Autor:innen und Institutionen

  • National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan.
  • Department of Electronics and Electrical Engineering, Faculty of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohoku-ku, Yokohama 223-8522, Japan.

ORCID-Profile: Erika Mori, Yue Qiu, Hirokatsu Kataoka, Yoshimitsu Aoki

Zitieren

APA

Mori, E., Qiu, Y., Kataoka, H., & Aoki, Y. (2025). A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans. Sensors. https://doi.org/10.3390/s25020477

Vancouver

Mori E, Qiu Y, Kataoka H, Aoki Y. A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans. Sensors. 2025. doi:10.3390/s25020477

BibTeX
@article{Mori2025,
  author = {Mori, Erika and Qiu, Yue and Kataoka, Hirokatsu and Aoki, Yoshimitsu},
  title = {A Comprehensive Analysis of a Social Intelligence Dataset and Response Tendencies Between Large Language Models (LLMs) and Humans},
  journal = {Sensors},
  year = {2025},
  doi = {10.3390/s25020477}
}
Kostenlos starten →