Surgical phase and instrument recognition: how to identify appropriate dataset splits
International Journal of Computer Assisted Radiology and Surgery · 2024
DOI: 10.1007/s11548-024-03063-9 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, 2024. Verfasst von Georgii Kostiuchik und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Cardiac Surgery, Heidelberg University Hospital, Heidelberg, Germany. georgii.kostiuchik@med.uni-heidelberg.de. Hauptthemen laut MEDLINE: Surgical Instruments, Machine Learning. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Klaus Tschira Stiftung.
Abstract (Originalsprache)
Purpose
Machine learning approaches can only be reliably evaluated if training, validation, and test data splits are representative and not affected by the absence of classes. Surgical workflow and instrument recognition are two tasks that are complicated in this manner, because of heavy data imbalances resulting from different length of phases and their potential erratic occurrences. Furthermore, sub-properties like instrument (co-)occurrence are usually not particularly considered when defining the split.
Methods
We present a publicly available data visualization tool that enables interactive exploration of dataset partitions for surgical phase and instrument recognition. The application focuses on the visualization of the occurrence of phases, phase transitions, instruments, and instrument combinations across sets. Particularly, it facilitates assessment of dataset splits, especially regarding identification of sub-optimal dataset splits.
Results
We performed analysis of the datasets Cholec80, CATARACTS, CaDIS, M2CAI-workflow, and M2CAI-tool using the proposed application. We were able to uncover phase transitions, individual instruments, and combinations of surgical instruments that were not represented in one of the sets. Addressing these issues, we identify possible improvements in the splits using our tool. A user study with ten participants demonstrated that the participants were able to successfully solve a selection of data exploration tasks.
Conclusion
In highly unbalanced class distributions, special care should be taken with respect to the selection of an appropriate dataset split because it can greatly influence the assessments of machine learning approaches. Our interactive tool allows for determination of better splits to improve current practices in the field. The live application is available at https://cardio-ai.github.io/endovis-ml/ .
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Cardiac Surgery, Heidelberg University Hospital, Heidelberg, Germany. georgii.kostiuchik@med.uni-heidelberg.de.
- DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim, Heidelberg, Germany. georgii.kostiuchik@med.uni-heidelberg.de.
- Department of Cardiac Surgery, Heidelberg University Hospital, Heidelberg, Germany.
- DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim, Heidelberg, Germany.
- Department of Simulation and Graphics, University of Magdeburg, Magdeburg, Germany.
- Department of Computer Science, Mannheim University of Applied Sciences, Mannheim, Germany.
Förderung
- Klaus Tschira Stiftung
Zitieren
APA
Kostiuchik, G., Sharan, L., Mayer, B., Wolf, I., Preim, B., & Engelhardt, S. (2024). Surgical phase and instrument recognition: how to identify appropriate dataset splits. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-024-03063-9
Vancouver
Kostiuchik G, Sharan L, Mayer B, Wolf I, Preim B, Engelhardt S. Surgical phase and instrument recognition: how to identify appropriate dataset splits. International Journal of Computer Assisted Radiology and Surgery. 2024. doi:10.1007/s11548-024-03063-9
BibTeX
@article{Kostiuchik2024,
author = {Kostiuchik, Georgii and Sharan, Lalith and Mayer, Benedikt and Wolf, Ivo and Preim, Bernhard and Engelhardt, Sandy},
title = {Surgical phase and instrument recognition: how to identify appropriate dataset splits},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2024},
doi = {10.1007/s11548-024-03063-9}
}
EUVIMED