Experience Filter: Translating Heterogeneous Signals into Motion
Advanced Science · 2026-11-10
DOI: 10.1002/advs.78602 ↗At a glance
Journal article in Advanced Science, published online on 2026-11-10. Written by Ekrem Yüksel and 2 co-authors. Institution: Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland. 43 references. The full text is freely available (open access, CC BY 4.0). Funded by: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung.
Abstract (original language)
Human experience is inherently rich and multisensory, composed of intertwined visual, auditory, and kinesthetic feedback. Digital media capture only its audiovisual dimensions, omitting its mechanosensory dimensions. While full‐body (kinesthetic) haptic feedback can enrich media, no general method exists for deriving structured motion from heterogeneous signals and combining it with the audiovisual content for rendering. The Experience Filter maps multimedia signals into structured motion trajectories through a common pipeline of modality‐specific feature extraction followed by a modality‐agnostic shared encoding, recombining them into a synchronized multisensory experience: each signal is reduced to a shared descriptor space – signal intensity, dominant spectral feature, and distance from neutral axis – then mapped to a trajectory via a fixed, deterministic, training‐free encoding. Because this encoding space is hardware‐independent, the motion can be retargeted to robotic platforms of arbitrary degrees of freedom and morphology. A feature‐sensitivity analysis shows that the descriptor axes are independent and the encoding‐space topology follows from the chosen descriptors rather than being fixed. The framework is demonstrated on real‐world music and images, producing continuous, synchronized image‐audio‐motion experiences without manual authoring. The Experience Filter thus provides an analytical mapping from multimedia signals to combined image‐audio‐motion rendering, enabling expressive, user‐adjustable experience across robotic platforms.
Subjects
Author keywords
Authors and institutions
- Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland.
ORCID profiles: Ekrem Yüksel, Alihan Bakir, Jamie Paik
Funding
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
Cite
APA
Yüksel, E., Bakir, A., & Paik, J. (2026). Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. https://doi.org/10.1002/advs.78602
Vancouver
Yüksel E, Bakir A, Paik J. Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. 2026. doi:10.1002/advs.78602
BibTeX
@article{Yksel2026,
author = {Yüksel, Ekrem and Bakir, Alihan and Paik, Jamie},
title = {Experience Filter: Translating Heterogeneous Signals into Motion},
journal = {Advanced Science},
year = {2026},
doi = {10.1002/advs.78602}
}
EUVIMED