EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

From multimodal features to behavioural inferences: A pipeline to model engagement in human-robot interactions

Soham Joshi iD, Arpitha Malavalli iD, Shrisha Rao iD

PLOS ONE · 08.11.2023 · 18 (11) · e0285749

DOI: 10.1371/journal.pone.0285749 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 08.11.2023. Verfasst von Soham Joshi und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, International Institute of Information Technology Bangalore, Bangalore, Karnataka, India. Hauptthemen laut MEDLINE: Interpersonal Relations, Robotics. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Modelling the engaging behaviour of humans using multimodal data collected during human-robot interactions has attracted much research interest. Most methods that have been proposed previously predict engaging behaviour directly from multimodal features, and do not incorporate personality inferences or any theories of interpersonal behaviour in human-human interactions. This work investigates whether personality inferences and attributes from interpersonal theories of behaviour (like attitude and emotion) further augment the modelling of engaging behaviour. We present a novel pipeline to model engaging behaviour that incorporates the Big Five personality traits, the Interpersonal Circumplex (IPC), and the Triandis Theory of Interpersonal Behaviour (TIB). We extract first-person vision and physiological features from the MHHRI dataset and predict the Big Five personality traits using a Support Vector Machine. Subsequently, we empirically validate the advantage of incorporating personality in modelling engaging behaviour and present a novel method that effectively uses the IPC to obtain scores for a human’s attitude and emotion from their Big Five traits. Finally, our results demonstrate that attitude and emotion are correlates of behaviour even in human-robot interactions, as suggested by the TIB for human-human interactions. Furthermore, incorporating the IPC and the Big Five traits helps generate behavioural inferences that supplement the engaging behaviour prediction, thus enriching the pipeline. Engagement modelling has a wide range of applications in domains like online learning platforms, assistive robotics, and intelligent conversational agents. Practitioners can also use this work in cognitive modelling and psychology to find more complex and subtle relations between humans’ behaviour and personality traits, and discover new dynamics of the human psyche. The code will be made available at: https://github.com/soham-joshi/engagement-prediction-mhhri .

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Interpersonal RelationsRoboticsHumansPersonalityEmotionsPersonality Disorders

Autor:innen und Institutionen

  • Department of Computer Science, International Institute of Information Technology Bangalore, Bangalore, Karnataka, India.

ORCID-Profile: Soham Joshi, Arpitha Malavalli, Shrisha Rao

Zitieren

APA

Joshi, S., Malavalli, A., & Rao, S. (2023). From multimodal features to behavioural inferences: A pipeline to model engagement in human-robot interactions. PLOS ONE, 18(11), e0285749. https://doi.org/10.1371/journal.pone.0285749

Vancouver

Joshi S, Malavalli A, Rao S. From multimodal features to behavioural inferences: A pipeline to model engagement in human-robot interactions. PLOS ONE. 2023;18(11):e0285749. doi:10.1371/journal.pone.0285749

BibTeX
@article{Joshi2023,
  author = {Joshi, Soham and Malavalli, Arpitha and Rao, Shrisha},
  title = {From multimodal features to behavioural inferences: A pipeline to model engagement in human-robot interactions},
  journal = {PLOS ONE},
  year = {2023},
  volume = {18},
  number = {11},
  pages = {e0285749},
  doi = {10.1371/journal.pone.0285749}
}
Kostenlos starten →