EUVIMEDEUVIMED entdecken
StudienprotokollOpen Access · CC BY 4.0

ROEP: A Robotics-Oriented Evaluation Protocol for Deployment-Facing Vision–Language–Action Manipulation Policies

Sangwoo Han iD, Hyunguk Choi iD

Sensors · 27.07.2026 · 26 (15) · 4757

DOI: 10.3390/s26154757 ↗

Kurz gefasst

Studienprotokoll in der Zeitschrift Sensors, online erschienen am 27.07.2026. Verfasst von Sangwoo Han und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Artificial Intelligence Software, Cheongju University, Cheongju 28503, Republic of Korea. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Chungbuk Regional Innovation System & Education Center, Ministry of Education (MOE) and the Chungcheongbuk-do, Republic of Korea.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision–Language–Action (VLA) policies are increasingly evaluated on language-conditioned robotic manipulation benchmarks, but success rate alone often obscures runtime-interface alignment, the repeatability of observation-degradation effects, and the recovery-relevant semantics of failures. This study proposes ROEP (Robotics-Oriented Evaluation Protocol), a VLA-targeted deployment-oriented evaluation protocol that converts rollout outcomes into claim-level evidence for closed-loop robotic manipulation. ROEP first verifies clean-condition evaluability and runtime fidelity of the sensor-to-action execution interface, then evaluates controlled visual perturbations, repeated-run reference variability, timeout-dominant failures, and recovery/shield support. We apply ROEP to OpenVLA, X-VLA, and VLA-Adapter on eight LIBERO Goal and Object tasks, producing a 24-row evaluation matrix with complete clean and medium-perturbation evidence. Under the evaluated LIBERO simulation setting, the results show that runtime-interface fidelity and checkpoint provenance are important for interpreting X-VLA and OpenVLA Object results, while VLA-Adapter maintains strong Goal-suite performance but exhibits a substantial Object-suite clean-to-perturbation drop dominated by timeout termination. ROEP therefore clarifies which deployment-relevant claims are supported, withheld, or insufficiently evidenced by the available rollouts without certifying open-world deployment safety or recovery success. The protocol complements existing VLA benchmarks by reporting success rate together with runtime fidelity, repeatability, failure semantics, and recovery/shield support evidence.

Schlagwörter

Stichwörter der Autor:innen

deployment-oriented evaluationmultimodal sensingrobotic manipulationrobustness evaluationvision–language–action

Autor:innen und Institutionen

  • Department of Artificial Intelligence Software, Cheongju University, Cheongju 28503, Republic of Korea.

ORCID-Profile: Sangwoo Han, Hyunguk Choi

Förderung

  • Chungbuk Regional Innovation System & Education Center
  • Ministry of Education (MOE) and the Chungcheongbuk-do, Republic of Korea

Zitieren

APA

Han, S., & Choi, H. (2026). ROEP: A Robotics-Oriented Evaluation Protocol for Deployment-Facing Vision–Language–Action Manipulation Policies. Sensors, 26(15), 4757. https://doi.org/10.3390/s26154757

Vancouver

Han S, Choi H. ROEP: A Robotics-Oriented Evaluation Protocol for Deployment-Facing Vision–Language–Action Manipulation Policies. Sensors. 2026;26(15):4757. doi:10.3390/s26154757

BibTeX
@article{Han2026,
  author = {Han, Sangwoo and Choi, Hyunguk},
  title = {ROEP: A Robotics-Oriented Evaluation Protocol for Deployment-Facing Vision–Language–Action Manipulation Policies},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {15},
  pages = {4757},
  doi = {10.3390/s26154757}
}
Kostenlos starten →