EUVIMEDEUVIMED entdecken
Crossref

Continuous Versus Discrete Tactile Representations for Vision-Language-Action Avocado Sorting

Ran Yang, Yuhao Lei, Lihong Yang, Yiming Feng

Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 950-958

DOI: 10.1609/aaaiss.v10i1.43512 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Ran Yang und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision-language-action (VLA) policies typically have no direct channel for non-visual physical properties, such as tactile ripeness: such evidence is either dropped entirely or folded into the policy as a discrete signal, a classifier output substituted into a text prompt. The latter reuses the language channel VLA models are pretrained on, but compresses a spatially structured deformation field into a low-bandwidth categorical cue represented by only a few language tokens. We investigate how the pathway used to incorporate tactile evidence affects contact-rich manipulation. Using robotic avocado ripeness sorting as a testbed, where tactile deformation cues are inaccessible to vision, we compare three conditions: a vision-only pi0 baseline with no tactile signal, discrete text-prompt conditioning on a separately trained tactile classifier, and continuous fusion, where GelSight tactile images are encoded via a pretrained DINOv2 encoder and concatenated with visual embeddings prior to the pi0 backbone, without added cross-modal attention. In a small-scale evaluation on 10 held-out avocados unseen during demonstration collection, with each specimen tested twice per method, continuous fusion reaches a 95% sorting success rate, versus 80% for discrete text-prompt conditioning and 70% with no tactile signal, at only 7.7% inference overhead over the vision-only baseline; the discrete classifier matches the specimen label in all 20 of its robot trials. The central observation is that even a correct tactile category can fail to control a pretrained action policy when communicated only through a prompt.

Zitieren

APA

Yang, R., Lei, Y., Yang, L., & Feng, Y. (2026). Continuous Versus Discrete Tactile Representations for Vision-Language-Action Avocado Sorting. Proceedings of the AAAI Symposium Series, 10(1), 950-958. https://doi.org/10.1609/aaaiss.v10i1.43512

Vancouver

Yang R, Lei Y, Yang L, Feng Y. Continuous Versus Discrete Tactile Representations for Vision-Language-Action Avocado Sorting. Proceedings of the AAAI Symposium Series. 2026;10(1):950-958. doi:10.1609/aaaiss.v10i1.43512

BibTeX
@article{Yang2026,
  author = {Yang, Ran and Lei, Yuhao and Yang, Lihong and Feng, Yiming},
  title = {Continuous Versus Discrete Tactile Representations for Vision-Language-Action Avocado Sorting},
  journal = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {950--958},
  doi = {10.1609/aaaiss.v10i1.43512}
}
Kostenlos starten →