EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Neuro-Symbolic Spatiotemporal Vision Transformer for Zero-Shot Safety Behavior Recognition

Rui Song, Yi Zhang, Fangzhou Hu, Ankai Zhang, Zhengwei Chang, Linghao Zhang

Applied Sciences · 12.11.2026 · 16 (22) · 11227

DOI: 10.3390/app162211227 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 12.11.2026. Verfasst von Rui Song und 5 weiteren Autor:innen. Die Arbeit nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: State Grid Corporation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study evaluates video-level behavior recognition on the seven-class Construction Meta Action (CMA) benchmark using a neuro-symbolic spatiotemporal Vision Transformer. An MViTv2-S backbone feeds a neural classification head and eight weakly supervised predicate variables, which are scored against fixed class prototypes; a temporal extension uses six log-domain operators over four ordered segments. On 540 official test videos, the fused model achieves 93.35% Macro-AP, compared with 93.23% for the neural branch. Zero-shot recognition is evaluated as leave-one-class-out transfer within CMA: the target class is excluded from task-specific head training and model selection, while Kinetics pretraining and target-class prototypes remain available. Mean held-out AP is 47.66% for the rule branch and 44.69% for the neural branch, without a statistically established difference. Log-domain computation maintains finite outputs and gradients under the reported stress conditions. The measured offline cost is 199.36 ms/video for single-clip file processing and approximately 1.17 s/video for four-segment extraction on an RTX 3080 Ti laptop GPU. The findings support inspection of prototype-based score construction and numerical behavior within the evaluated benchmark. Predicate semantics are not independently annotated, predictions are not attributed to individual workers, and cross-site generalization and live-stream performance have not been validated.

Förderung

  • State Grid Corporation of China

Zitieren

APA

Song, R., Zhang, Y., Hu, F., Zhang, A., Chang, Z., & Zhang, L. (2026). A Neuro-Symbolic Spatiotemporal Vision Transformer for Zero-Shot Safety Behavior Recognition. Applied Sciences, 16(22), 11227. https://doi.org/10.3390/app162211227

Vancouver

Song R, Zhang Y, Hu F, Zhang A, Chang Z, Zhang L. A Neuro-Symbolic Spatiotemporal Vision Transformer for Zero-Shot Safety Behavior Recognition. Applied Sciences. 2026;16(22):11227. doi:10.3390/app162211227

BibTeX
@article{Song2026,
  author = {Song, Rui and Zhang, Yi and Hu, Fangzhou and Zhang, Ankai and Chang, Zhengwei and Zhang, Linghao},
  title = {A Neuro-Symbolic Spatiotemporal Vision Transformer for Zero-Shot Safety Behavior Recognition},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {22},
  pages = {11227},
  doi = {10.3390/app162211227}
}
Kostenlos starten →