Sketching with Hardware Lab: The SoundSense — Human Machine Interaction Beyond the Screen
Technischen Universität Darmstadt · 15.09.2026
DOI: 10.26083/tuda-8333 ↗Kurz gefasst
Fachartikel in der Zeitschrift Technischen Universität Darmstadt, erschienen am 15.09.2026. Verfasst von Rohit Jain.
Abstract (Originalsprache)
There have been many recent advances in the field of Human Computer Interaction (HCI) with most solutions for computer output to humans being centered around the Displays and the Screens of various sizes and types. These solutions use an approach of various different types of intuitive user interfaces with text and graphics to render various types of information to the users in large volume. In our lab, we pursued an alternative approach to explore other methods to output information to the human users beyond screens, that is without using any display or screen. With this goal in mind, we developed a lab project from scratch, which could sense ambient sounds and noises in an environment and employ machine learning-based classification to detect its type (with a certain level of accuracy), and convey this information over to the user in the nearby proximity through wireless channel. For this purpose, we used a Raspberry Pi 4 based Base-Station for sound capture and processing, along with enabling a Wi-Fi hotspot for the wireless connectivity. The user receives this signal (corresponding to the type of noise or sound) through a wearable band type gadget with the signal being rendered through a certain type of sensations on the skin. There are three type of sensations which are delivered through the wearable band: Rubbing, Poking and Pressing. A different combination of all three types is used to convey (or classify) a specific type of noise or sound in the environment such as doorbell, baby crying, etc. Up to 5 wearable bands can be connected to a receiving station through individual cables, all of which can be fit into a clothing apparel through multiple wearable gadget bands which we call The SoundSense Suit, and it can be powered by a 5V-12V external battery or power supply. For this purpose, we used Raspberry Pi Pico 2W (RP2350) micro-controller with a servo motors control system. For the flexibility and wearing comfort of the user, we provided an option to conveniently adjust wearing the bands on different parts of body such as wrists, biceps, thighs, calves, head (inside a cap or helmet) or perhaps stomach or chest, maybe even underarms! To overcome the problem of dangling wires from receiver station connecting to the wearable bands, we recommend stitching the thin cables directly through the clothing apparel such as jacket, pants, jumpsuit, etc. With our initially developed prototype, a user can conveniently discern more than 8 different classified audio signals per wearable band. So, with a combination 5 wearable bands more than 40 distinct audio signals can be identified or perceived. These skin sensation signals can be memorized with user training for different purposes. Furthermore, with our Wi-Fi hotspot connectivity system we can easily extend our product to attach and work with multiple receiver devices of same or different actuation type, such as a LED panel or something better. This which would enable us to create a business ecosystem of different types of product accessories for prospective customers. With the wear-ability aspect of this product, there is an option to promote this product through business tie-up with different clothing fashion brand corporations, for advertising it as a fashion style trend along with use as a personal utility gadget.
Schlagwörter
Zitieren
APA
Jain, R. (2026). Sketching with Hardware Lab: The SoundSense — Human Machine Interaction Beyond the Screen. Technischen Universität Darmstadt. https://doi.org/10.26083/tuda-8333
Vancouver
Jain R. Sketching with Hardware Lab: The SoundSense — Human Machine Interaction Beyond the Screen. Technischen Universität Darmstadt. 2026. doi:10.26083/tuda-8333
BibTeX
@article{Jain2026,
author = {Jain, Rohit},
title = {Sketching with Hardware Lab: The SoundSense — Human Machine Interaction Beyond the Screen},
journal = {Technischen Universität Darmstadt},
year = {2026},
doi = {10.26083/tuda-8333}
}
EUVIMED