Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment
Bioengineering · 29.05.2026 · 13 (6) · 640
DOI: 10.3390/bioengineering13060640 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioengineering, online erschienen am 29.05.2026. Verfasst von Taiyo Ikeido und 11 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Miki Haseyama, Takahiro Ogawa, Ren Togo.
Abstract (Originalsprache)
Surgical workflow understanding requires recognizing procedural phases and fine-grained activities from long-horizon videos, yet acquiring dense annotations for surgical video analysis is costly and requires medical expertise. To address this challenge, we present a text-guided and annotation-efficient framework for surgical video understanding based on a frozen surgical vision–language-pretrained (VLP) encoder and a lightweight temporal adapter. The frozen SurgVLP image encoder provides frame-level visual embeddings, and the temporal adapter aggregates them into clip-level representations while preserving compatibility with the pretrained visual–text embedding space. We evaluate the proposed framework on CholecT50 using text-guided prototype matching for phase recognition and few-shot triplet recognition. Experiments show that temporal adaptation improves phase recognition while preserving the pretrained SurgVLP embedding space. In particular, among the evaluated methods, the proposed Text Contrastive method with rich phase prompts achieves the highest phase recognition performance, outperforming the phase-only baseline. Furthermore, the proposed framework enables classifier-free few-shot triplet recognition in the frozen text space without training a dedicated triplet classifier. These results suggest that effective surgical video understanding under limited annotation depends not only on temporal adaptation but also on preserving alignment with the pretrained text space and using semantically informative text prompts.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Graduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.
- Faculty of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.
- Department of Neurosurgery, Graduate School of Medicine, Hokkaido University, Kita 15, Nishi 7, Kita-ku, Sapporo 060-8638, Hokkaido, Japan.
- Department of Gastroenterological Surgery II, Graduate School of Medicine, Hokkaido University, Kita 15, Nishi 7, Kita-ku, Sapporo 060-8638, Hokkaido, Japan.
- Faculty of Health Sciences, Hokkaido University, Kita 12, Nishi 5, Kita-ku, Sapporo 060-0812, Hokkaido, Japan.
- Department of Diagnostic Imaging, Graduate School of Medicine, Hokkaido University, Kita 15, Nishi 7, Kita-ku, Sapporo 060-8638, Hokkaido, Japan.
ORCID-Profile: Taiyo Ikeido, Ren Togo, Takahiro Ogawa, Taku Sugiyama, Saseem Poudel, Hiroyuki Sugimori, Minghui Tang, Feng Han, Kenji Hirata, Kohsuke Kudo, Miki Haseyama
Förderung
- Miki Haseyama
- Takahiro Ogawa
- Ren Togo
Zitieren
APA
Ikeido, T., Togo, R., Ogawa, T., Sugiyama, T., Poudel, S., Sugimori, H., Tang, M., Han, F., Koyano, H., Hirata, K., Kudo, K., & Haseyama, M. (2026). Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment. Bioengineering, 13(6), 640. https://doi.org/10.3390/bioengineering13060640
Vancouver
Ikeido T, Togo R, Ogawa T, Sugiyama T, Poudel S, Sugimori H, et al. Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment. Bioengineering. 2026;13(6):640. doi:10.3390/bioengineering13060640
BibTeX
@article{Ikeido2026,
author = {Ikeido, Taiyo and Togo, Ren and Ogawa, Takahiro and Sugiyama, Taku and Poudel, Saseem and Sugimori, Hiroyuki and Tang, Minghui and Han, Feng and Koyano, Hidenori and Hirata, Kenji and Kudo, Kohsuke and Haseyama, Miki},
title = {Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment},
journal = {Bioengineering},
year = {2026},
volume = {13},
number = {6},
pages = {640},
doi = {10.3390/bioengineering13060640}
}
EUVIMED