EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition

Zikang Leng, Amitrajit Bhattacharjee, Hrudhai Rajasekhar, Lizhe Zhang, Elizabeth Bruda, Hyeokhyen Kwon, Thomas Plötz

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 22.08.2024 · 8 (3) · 1-32

DOI: 10.1145/3678545 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 09.09.2024. Verfasst von Zikang Leng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 67-mal zitiert (Crossref) und nennt 83 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

One of the primary challenges in the field of human activity recognition (HAR) is the lack of large labeled datasets. This hinders the development of robust and generalizable models. Recently, cross modality transfer approaches have been explored that can alleviate the problem of data scarcity. These approaches convert existing datasets from a source modality, such as video, to a target modality, such as inertial measurement units (IMUs). With the emergence of generative AI models such as large language models (LLMs) and text-driven motion synthesis models, language has become a promising source data modality as well - as shown in proof of concepts such as IMUGPT. In this work, we conduct a large-scale evaluation of language-based cross modality transfer to determine their effectiveness for HAR. Based on this study, we introduce two new extensions for IMUGPT that enhance its use for practical HAR application scenarios: a motion filter capable of filtering out irrelevant motion sequences to ensure the relevance of the generated virtual IMU data, and a set of metrics that measure the diversity of the generated data facilitating the determination of when to stop generating virtual IMU data for both effective and efficient processing. We demonstrate that our diversity metrics can reduce the effort needed for the generation of virtual IMU data by at least 50%, which opens up IMUGPT for practical use cases beyond a mere proof of concept.

Zitieren

APA

Leng, Z., Bhattacharjee, A., Rajasekhar, H., Zhang, L., Bruda, E., Kwon, H., & Plötz, T. (2024). IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(3), 1-32. https://doi.org/10.1145/3678545

Vancouver

Leng Z, Bhattacharjee A, Rajasekhar H, Zhang L, Bruda E, Kwon H, et al. IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2024;8(3):1-32. doi:10.1145/3678545

BibTeX
@article{Leng2024,
  author = {Leng, Zikang and Bhattacharjee, Amitrajit and Rajasekhar, Hrudhai and Zhang, Lizhe and Bruda, Elizabeth and Kwon, Hyeokhyen and Plötz, Thomas},
  title = {IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition},
  journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  year = {2024},
  volume = {8},
  number = {3},
  pages = {1--32},
  doi = {10.1145/3678545}
}
Kostenlos starten →