The EarSAVAS Dataset
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 13.05.2024 · 8 (2) · 1-26
DOI: 10.1145/3659616 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 15.05.2024. Verfasst von Xiyuxing Zhang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of China, Young Elite Scientists Sponsorship Program by CAST, Tsinghua University Initiative Scientifc Research Program, Beijing Key Lab of Networked Multimedia, Institute for Artificial Intelligence, Tsinghua University.
Abstract (Originalsprache)
Subject-aware vocal activity sensing on wearables, which specifically recognizes and monitors the wearer's distinct vocal activities, is essential in advancing personal health monitoring and enabling context-aware applications. While recent advancements in earables present new opportunities, the absence of relevant datasets and effective methods remains a significant challenge. In this paper, we introduce EarSAVAS, the first publicly available dataset constructed specifically for subject-aware human vocal activity sensing on earables. EarSAVAS encompasses eight distinct vocal activities from both the earphone wearer and bystanders, including synchronous two-channel audio and motion data collected from 42 participants totaling 44.5 hours. Further, we propose EarVAS, a lightweight multi-modal deep learning architecture that enables efficient subject-aware vocal activity recognition on earables. To validate the reliability of EarSAVAS and the efficiency of EarVAS, we implemented two advanced benchmark models. Evaluation results on EarSAVAS reveal EarVAS's effectiveness with an accuracy of 90.84% and a Macro-AUC of 89.03%. Comprehensive ablation experiments were conducted on benchmark models and demonstrated the effectiveness of feedback microphone audio and highlighted the potential value of sensor fusion in subject-aware vocal activity sensing on earables. We hope that the proposed EarSAVAS and benchmark models can inspire other researchers to further explore efficient subject-aware human vocal activity sensing on earables.
Förderung
- Natural Science Foundation of China
- Young Elite Scientists Sponsorship Program by CAST
- Tsinghua University Initiative Scientifc Research Program, Beijing Key Lab of Networked Multimedia, Institute for Artificial Intelligence, Tsinghua University
- Undergraduate / Graduate Education Innovation Grants, Tsinghua University, and Beijing National Research Center for Information Science and Technology
Zitieren
APA
Zhang, X., Wang, Y., Han, Y., Liang, C., Chatterjee, I., Tang, J., Yi, X., Patel, S., & Shi, Y. (2024). The EarSAVAS Dataset. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(2), 1-26. https://doi.org/10.1145/3659616
Vancouver
Zhang X, Wang Y, Han Y, Liang C, Chatterjee I, Tang J, et al. The EarSAVAS Dataset. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2024;8(2):1-26. doi:10.1145/3659616
BibTeX
@article{Zhang2024,
author = {Zhang, Xiyuxing and Wang, Yuntao and Han, Yuxuan and Liang, Chen and Chatterjee, Ishan and Tang, Jiankai and Yi, Xin and Patel, Shwetak and Shi, Yuanchun},
title = {The EarSAVAS Dataset},
journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
year = {2024},
volume = {8},
number = {2},
pages = {1--26},
doi = {10.1145/3659616}
}
EUVIMED