EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Exploration of Pointing Forms in Customer–Shopkeeper Interactions

Yongqiang Jiang, Malcolm Doering, Takayuki Kanda

ACM Transactions on Human-Robot Interaction · 31.07.2026 · 15 (4) · 1-28

DOI: 10.1145/3811026 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Human-Robot Interaction, online erschienen am 15.06.2026. Verfasst von Yongqiang Jiang und 2 weiteren Autor:innen. Die Arbeit nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JST Moonshot R and D, JSPS KAKENHI, CSC scholarship.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

To investigate the use of different pointing forms in service scenarios, we collected the ShopPoint dataset, a skeleton-based dataset of pointing gestures from customer–shopkeeper interactions in a camera shop scenario. Thirteen participants took part in the data collection, including 3 shopkeepers with real-world customer service experience and 10 customers. We recorded 61 one-to-one role-played interactions. Coders annotated pointing gestures from videos of these interactions, emphasizing pointing arm forms (straight-arm, bent-arm, and hand-only pointing) and hand forms (index-finger and open-hand pointing). This annotation process resulted in 2,959 pointing gestures. We conducted statistical analysis on the annotated data. The analysis revealed that bent-arm pointing was used more frequently than other arm forms. Straight-arm pointing was used more for far targets than for close targets, and hand-only was used more for close targets. Shopkeepers used bent-arm pointing more frequently than customers when referring to far targets. To evaluate the recognition of these pointing gestures, we tested several existing Skeleton-based Action Recognition (SAR) methods on the dataset. The highest accuracy was achieved at 72.51% by using transfer learning (i.e., pretraining and fine-tuning). This evaluation indicates that though transfer learning aids performance, recognizing pointing with diverse forms remains challenging.

Förderung

  • JST Moonshot R and D
  • JSPS KAKENHI
  • CSC scholarship

Zitieren

APA

Jiang, Y., Doering, M., & Kanda, T. (2026). Exploration of Pointing Forms in Customer–Shopkeeper Interactions. ACM Transactions on Human-Robot Interaction, 15(4), 1-28. https://doi.org/10.1145/3811026

Vancouver

Jiang Y, Doering M, Kanda T. Exploration of Pointing Forms in Customer–Shopkeeper Interactions. ACM Transactions on Human-Robot Interaction. 2026;15(4):1-28. doi:10.1145/3811026

BibTeX
@article{Jiang2026,
  author = {Jiang, Yongqiang and Doering, Malcolm and Kanda, Takayuki},
  title = {Exploration of Pointing Forms in Customer–Shopkeeper Interactions},
  journal = {ACM Transactions on Human-Robot Interaction},
  year = {2026},
  volume = {15},
  number = {4},
  pages = {1--28},
  doi = {10.1145/3811026}
}
Kostenlos starten →