Uncertainty-Aware Topological Persistence Guided Knowledge Distillation on Wearable Sensor Data
IEEE Internet of Things Journal · 15.09.2024 · 11 (18) · 30413-30429
DOI: 10.1109/jiot.2024.3412980 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Internet of Things Journal, erschienen am 15.09.2024. Verfasst von Eun Som Jeon und 2 weiteren Autor:innen. Beteiligte Einrichtung: Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe…. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 78 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIGMS NIH HHS, National Institutes of Health, Joint DMS/NIGMS Initiative to Support Research at the Interface of the Biological and Mathematical Sciences.
Abstract (Originalsprache)
In applications involving analysis of wearable sensor data, machine learning techniques that use features from topological data analysis (TDA) have demonstrated remarkable performance. Persistence images (PIs) generated through TDA prove effective in capturing robust features, especially to signal perturbations, thus complementing classical time-series features. Despite its promising performance, utilizing TDA to create PI entails significant computational resources and time, posing challenges for applications on small devices. Knowledge distillation (KD) emerges as a solution to address these challenges, as it can produce a compact model. Using multiple teachers one trained with raw time-series and another with topological features, is a viable approach to distill a single compact student model. In such a case, the two teachers will have different statistical characteristics and need some form of feature harmonization. To tackle these issues, we propose uncertainty-aware topological persistence guided knowledge distillation. This approach involves separating common and distinct components between teachers and applying varying weights to control their effects. To enhance the knowledge provided to a student, uncertain features from teachers are rectified using uncertainty scores. We leverage feature similarities to offer more valuable information and employ relationships computed based on orthogonal properties to prevent excessive feature transformation. Ultimately, our method yields a robust single student that operates solely on time-series data at test-time. We validate the effectiveness of the proposed approach through empirical evaluations across various combinations of models and datasets, demonstrating its robustness and efficacy in different scenarios. The proposed method enhances the classification performance of a student model by approximately 4.3% compared to a model learned from scratch on GENEActiv.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ 85281 USA.
- College of Health Solutions, Arizona State University, Phoenix, AZ 85004 USA.
Förderung
- NIGMS NIH HHS
- National Institutes of Health
- Joint DMS/NIGMS Initiative to Support Research at the Interface of the Biological and Mathematical Sciences
- NSF
Zitieren
APA
Jeon, E. S., Buman, M. P., & Turaga, P. (2024). Uncertainty-Aware Topological Persistence Guided Knowledge Distillation on Wearable Sensor Data. IEEE Internet of Things Journal, 11(18), 30413-30429. https://doi.org/10.1109/jiot.2024.3412980
Vancouver
Jeon ES, Buman MP, Turaga P. Uncertainty-Aware Topological Persistence Guided Knowledge Distillation on Wearable Sensor Data. IEEE Internet of Things Journal. 2024;11(18):30413-30429. doi:10.1109/jiot.2024.3412980
BibTeX
@article{Jeon2024,
author = {Jeon, Eun Som and Buman, Matthew P and Turaga, Pavan},
title = {Uncertainty-Aware Topological Persistence Guided Knowledge Distillation on Wearable Sensor Data},
journal = {IEEE Internet of Things Journal},
year = {2024},
volume = {11},
number = {18},
pages = {30413--30429},
doi = {10.1109/jiot.2024.3412980}
}
EUVIMED