Where are they looking in the operating room?
International Journal of Computer Assisted Radiology and Surgery · 06.04.2026 · 21 (5) · 939-948
DOI: 10.1007/s11548-026-03604-4 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, online erschienen am 06.04.2026. Verfasst von Keqi Chen und 7 weiteren Autor:innen. Beteiligte Einrichtung: CNRS, INSERM, ICube, UMR7357, University of Strasbourg, 67000, Strasbourg, France. keqi.chen@unistra.fr. Hauptthemen laut MEDLINE: Operating Rooms, Fixation, Ocular, Attention. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 43 Literaturangaben. Gefördert von: ANR, Bpifrance, RUTH & ARTHUR SCHERBARTH STIFTUNG.
Abstract (Originalsprache)
PURPOSE: Gaze-following, the task of inferring where individuals are looking, has been widely studied in computer vision, advancing research in visual attention modeling, social scene understanding, and human-robot interaction. However, gaze-following has never been explored in the operating room (OR), a complex, high-stakes environment where visual attention plays an important role in surgical workflow analysis. In this work, we introduce the concept of gaze-following to the surgical domain, and demonstrate its great potential for understanding clinical roles, surgical phases, and team communications in the OR. METHODS: We extend the 4D-OR dataset with gaze-following annotations, and extend the Team-OR dataset with gaze-following and a new team communication activity annotations. Then, we propose novel approaches to address clinical role prediction, surgical phase recognition, and team communication detection using a gaze-following model. For role and phase recognition, we propose a gaze heatmap-based approach that uses gaze predictions solely; for team communication detection, we train a spatial-temporal model in a self-supervised way that encodes gaze-based clip features, and then feed the features into a temporal activity detection model. RESULTS: Experimental results on the 4D-OR and Team-OR datasets demonstrate that our approach achieves state-of-the-art performance on all downstream tasks. Quantitatively, our approach obtains F1 scores of 0.92 for clinical role prediction and 0.95 for surgical phase recognition. Furthermore, it significantly outperforms existing baselines in team communication detection, improving previous best performances by over 30%. CONCLUSION: We introduce gaze-following in the OR as a novel research direction in surgical data science, highlighting its great potential to advance surgical workflow analysis in computer-assisted interventions. Although limited to monocular 2D gaze prediction relying on manual annotations, our research clearly demonstrates the clinical value of gaze analysis from ceiling-mounted cameras. Future work will explore semantic understanding, multi-view learning, and few-shot approaches to further improve scalability and robustness.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- CNRS, INSERM, ICube, UMR7357, University of Strasbourg, 67000, Strasbourg, France. keqi.chen@unistra.fr.
- CNRS, INSERM, ICube, UMR7357, University of Strasbourg, 67000, Strasbourg, France.
- Department for Biomedical Research (DBMR), University of Bern, 3008, Bern, Switzerland.
- Department for Visceral Surgery and Medicine, Bern University Hospital, University of Bern, 3010, Bern, Switzerland.
- IHU Strasbourg, 67000, Strasbourg, France.
- University Hospital of Strasbourg, 67000, Strasbourg, France.
Förderung
- ANR
- Bpifrance
- RUTH & ARTHUR SCHERBARTH STIFTUNG
Zitieren
APA
Chen, K., Baributsa, S., Schewski, L., Srivastav, V., Mutter, D., Beldi, G., Keller, S., & Padoy, N. (2026). Where are they looking in the operating room?. International Journal of Computer Assisted Radiology and Surgery, 21(5), 939-948. https://doi.org/10.1007/s11548-026-03604-4
Vancouver
Chen K, Baributsa S, Schewski L, Srivastav V, Mutter D, Beldi G, et al. Where are they looking in the operating room?. International Journal of Computer Assisted Radiology and Surgery. 2026;21(5):939-948. doi:10.1007/s11548-026-03604-4
BibTeX
@article{Chen2026,
author = {Chen, Keqi and Baributsa, Séraphin and Schewski, Lilien and Srivastav, Vinkle and Mutter, Didier and Beldi, Guido and Keller, Sandra and Padoy, Nicolas},
title = {Where are they looking in the operating room?},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2026},
volume = {21},
number = {5},
pages = {939--948},
doi = {10.1007/s11548-026-03604-4}
}
EUVIMED