EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment

Taiyo Ikeido iD, Ren Togo iD, Takahiro Ogawa iD, Taku Sugiyama iD, Saseem Poudel iD, Hiroyuki Sugimori iD, Minghui Tang iD, Feng Han iD, Hidenori Koyano, Kenji Hirata iD, Kohsuke Kudo iD, Miki Haseyama iD

Bioengineering · 29.05.2026 · 13 (6) · 640

DOI: 10.3390/bioengineering13060640 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioengineering, online erschienen am 29.05.2026. Verfasst von Taiyo Ikeido und 11 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Miki Haseyama, Takahiro Ogawa, Ren Togo.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Surgical workflow understanding requires recognizing procedural phases and fine-grained activities from long-horizon videos, yet acquiring dense annotations for surgical video analysis is costly and requires medical expertise. To address this challenge, we present a text-guided and annotation-efficient framework for surgical video understanding based on a frozen surgical vision–language-pretrained (VLP) encoder and a lightweight temporal adapter. The frozen SurgVLP image encoder provides frame-level visual embeddings, and the temporal adapter aggregates them into clip-level representations while preserving compatibility with the pretrained visual–text embedding space. We evaluate the proposed framework on CholecT50 using text-guided prototype matching for phase recognition and few-shot triplet recognition. Experiments show that temporal adaptation improves phase recognition while preserving the pretrained SurgVLP embedding space. In particular, among the evaluated methods, the proposed Text Contrastive method with rich phase prompts achieves the highest phase recognition performance, outperforming the phase-only baseline. Furthermore, the proposed framework enables classifier-free few-shot triplet recognition in the frozen text space without training a dedicated triplet classifier. These results suggest that effective surgical video understanding under limited annotation depends not only on temporal adaptation but also on preserving alignment with the pretrained text space and using semantically informative text prompts.

Schlagwörter

Stichwörter der Autor:innen

CholecT50few-shot recognitionsurgical video understandingtemporal adaptationtext prototype matchingvision–language pretraining

Autor:innen und Institutionen

  • Graduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.
  • Faculty of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.
  • Department of Neurosurgery, Graduate School of Medicine, Hokkaido University, Kita 15, Nishi 7, Kita-ku, Sapporo 060-8638, Hokkaido, Japan.
  • Department of Gastroenterological Surgery II, Graduate School of Medicine, Hokkaido University, Kita 15, Nishi 7, Kita-ku, Sapporo 060-8638, Hokkaido, Japan.
  • Faculty of Health Sciences, Hokkaido University, Kita 12, Nishi 5, Kita-ku, Sapporo 060-0812, Hokkaido, Japan.
  • Department of Diagnostic Imaging, Graduate School of Medicine, Hokkaido University, Kita 15, Nishi 7, Kita-ku, Sapporo 060-8638, Hokkaido, Japan.

ORCID-Profile: Taiyo Ikeido, Ren Togo, Takahiro Ogawa, Taku Sugiyama, Saseem Poudel, Hiroyuki Sugimori, Minghui Tang, Feng Han, Kenji Hirata, Kohsuke Kudo, Miki Haseyama

Förderung

  • Miki Haseyama
  • Takahiro Ogawa
  • Ren Togo

Zitieren

APA

Ikeido, T., Togo, R., Ogawa, T., Sugiyama, T., Poudel, S., Sugimori, H., Tang, M., Han, F., Koyano, H., Hirata, K., Kudo, K., & Haseyama, M. (2026). Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment. Bioengineering, 13(6), 640. https://doi.org/10.3390/bioengineering13060640

Vancouver

Ikeido T, Togo R, Ogawa T, Sugiyama T, Poudel S, Sugimori H, et al. Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment. Bioengineering. 2026;13(6):640. doi:10.3390/bioengineering13060640

BibTeX
@article{Ikeido2026,
  author = {Ikeido, Taiyo and Togo, Ren and Ogawa, Takahiro and Sugiyama, Taku and Poudel, Saseem and Sugimori, Hiroyuki and Tang, Minghui and Han, Feng and Koyano, Hidenori and Hirata, Kenji and Kudo, Kohsuke and Haseyama, Miki},
  title = {Surgical Video Understanding with Alignment-Preserving Temporal Adaptation and Action Triplet Text Alignment},
  journal = {Bioengineering},
  year = {2026},
  volume = {13},
  number = {6},
  pages = {640},
  doi = {10.3390/bioengineering13060640}
}
Kostenlos starten →