EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition

Baiyu Chen, Wilson Wongso, Zechen Li, Yonchanok Khaokaew, Hao Xue, Flora D. Salim

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 15.06.2026 · 10 (2) · 1-29

DOI: 10.1145/3810218 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 15.06.2026. Verfasst von Baiyu Chen und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: ARC Centre of Excellence for Automated Decision-Making and Society.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The goal of creating intelligent, human-centered wearable systems for continuous activity understanding faces a fundamental trade-off: Egocentric video-based models capture rich semantic information and have demonstrated strong performance in human activity recognition (HAR), but their high power consumption, privacy concerns, and dependence on lighting limit their feasibility for continuous on-device recognition. In contrast, inertial measurement unit (IMU) sensors offer an energy-efficient, privacy-preserving alternative, yet lack large-scale annotated datasets, leading to weaker generalization. To bridge this gap, we propose COMODO, a cross-modal self-supervised distillation framework that transfers semantic knowledge from video to IMU without requiring labels. COMODO leverages a pretrained and frozen video encoder to construct a dynamic instance queue to align the feature distributions of video and IMU embeddings. This enables the IMU encoder to inherit rich semantic structure from video while maintaining its efficiency for real-world applications. Experiments on multiple egocentric HAR datasets show that COMODO consistently improves downstream performance, matching or surpassing fully supervised models, and demonstrating strong cross-dataset generalization. Benefiting from its simplicity and flexibility, COMODO is compatible with diverse pretrained video and time-series models, offering the potential to leverage more powerful teacher and student foundation models in future ubiquitous computing research. The code is available at this repository: https://github.com/cruiseresearchgroup/COMODO.

Förderung

  • ARC Centre of Excellence for Automated Decision-Making and Society

Zitieren

APA

Chen, B., Wongso, W., Li, Z., Khaokaew, Y., Xue, H., & Salim, F. D. (2026). COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 10(2), 1-29. https://doi.org/10.1145/3810218

Vancouver

Chen B, Wongso W, Li Z, Khaokaew Y, Xue H, Salim FD. COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2026;10(2):1-29. doi:10.1145/3810218

BibTeX
@article{Chen2026,
  author = {Chen, Baiyu and Wongso, Wilson and Li, Zechen and Khaokaew, Yonchanok and Xue, Hao and Salim, Flora D.},
  title = {COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition},
  journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  year = {2026},
  volume = {10},
  number = {2},
  pages = {1--29},
  doi = {10.1145/3810218}
}
Kostenlos starten →