EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Causal Inference and Text-Assisted Localization for Medical Video Localization

Shikai Zhou, Tingting Bo, Zibo Xu, Bingyang Zhou, Shaopeng Zhang

IEEE Access · 2026 · 14 · 59095-59108

DOI: 10.1109/access.2026.3684826 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Shikai Zhou und 4 weiteren Autor:innen. Die Arbeit nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Tianjin.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video localization aims to identify the temporal segment within a video that best corresponds to a given textual query. In medical scenarios, however, the visual–textual relationship is substantially influenced by complex and often unobserved confounders arising from anatomical variability, procedure-specific motions, and heterogeneous imaging conditions. As a result, conventional localization methods struggle to establish reliable semantic alignment. To address these challenges, we propose a causal-interference-resilient framework specifically designed for medical video localization. The method introduces three complementary components. First, a causal inference module explicitly models the unobservability of latent confounders and applies a front-door adjustment strategy to recover unbiased visual–textual associations, thereby improving the causal validity of localization. Second, we incorporate a Medical Semantic Prior module tailored to the characteristics of the Kvasir-Capsule dataset. Capsule endoscopy videos exhibit domain-unique patterns such as intestinal lumen geometry, mucosal texture distributions, and subtle lesion morphology. By integrating structured anatomical priors and lesion-relevant cues into the representation learning process, the module constrains feature extraction toward clinically meaningful causal pathways rather than spurious correlations. Third, a BERT-based text-assisted localization module with attention pooling captures fine-grained linguistic semantics, enabling precise alignment between medical expressions and causally relevant visual events. Extensive experiments on five benchmark datasets, including MedVidQA and Kvasir-Capsule, demonstrate that the proposed method achieves superior localization performance. The framework ranked first in the TRECVID 2023 MedVidQA task and shows strong robustness under weak supervision when combined with CLIP-derived frame annotations. These findings validate the causal soundness and domain effectiveness of our approach.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Tianjin

Zitieren

APA

Zhou, S., Bo, T., Xu, Z., Zhou, B., & Zhang, S. (2026). Causal Inference and Text-Assisted Localization for Medical Video Localization. IEEE Access, 14, 59095-59108. https://doi.org/10.1109/access.2026.3684826

Vancouver

Zhou S, Bo T, Xu Z, Zhou B, Zhang S. Causal Inference and Text-Assisted Localization for Medical Video Localization. IEEE Access. 2026;14:59095-59108. doi:10.1109/access.2026.3684826

BibTeX
@article{Zhou2026,
  author = {Zhou, Shikai and Bo, Tingting and Xu, Zibo and Zhou, Bingyang and Zhang, Shaopeng},
  title = {Causal Inference and Text-Assisted Localization for Medical Video Localization},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {59095--59108},
  doi = {10.1109/access.2026.3684826}
}
Kostenlos starten →