Experience Filter: Translating Heterogeneous Signals into Motion
Advanced Science · 10.11.2026
DOI: 10.1002/advs.78602 ↗Kurz gefasst
Fachartikel in der Zeitschrift Advanced Science, online erschienen am 10.11.2026. Verfasst von Ekrem Yüksel und 2 weiteren Autor:innen. Beteiligte Einrichtung: Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung.
Abstract (Originalsprache)
Human experience is inherently rich and multisensory, composed of intertwined visual, auditory, and kinesthetic feedback. Digital media capture only its audiovisual dimensions, omitting its mechanosensory dimensions. While full‐body (kinesthetic) haptic feedback can enrich media, no general method exists for deriving structured motion from heterogeneous signals and combining it with the audiovisual content for rendering. The Experience Filter maps multimedia signals into structured motion trajectories through a common pipeline of modality‐specific feature extraction followed by a modality‐agnostic shared encoding, recombining them into a synchronized multisensory experience: each signal is reduced to a shared descriptor space – signal intensity, dominant spectral feature, and distance from neutral axis – then mapped to a trajectory via a fixed, deterministic, training‐free encoding. Because this encoding space is hardware‐independent, the motion can be retargeted to robotic platforms of arbitrary degrees of freedom and morphology. A feature‐sensitivity analysis shows that the descriptor axes are independent and the encoding‐space topology follows from the chosen descriptors rather than being fixed. The framework is demonstrated on real‐world music and images, producing continuous, synchronized image‐audio‐motion experiences without manual authoring. The Experience Filter thus provides an analytical mapping from multimedia signals to combined image‐audio‐motion rendering, enabling expressive, user‐adjustable experience across robotic platforms.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland.
ORCID-Profile: Ekrem Yüksel, Alihan Bakir, Jamie Paik
Förderung
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
Zitieren
APA
Yüksel, E., Bakir, A., & Paik, J. (2026). Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. https://doi.org/10.1002/advs.78602
Vancouver
Yüksel E, Bakir A, Paik J. Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. 2026. doi:10.1002/advs.78602
BibTeX
@article{Yksel2026,
author = {Yüksel, Ekrem and Bakir, Alihan and Paik, Jamie},
title = {Experience Filter: Translating Heterogeneous Signals into Motion},
journal = {Advanced Science},
year = {2026},
doi = {10.1002/advs.78602}
}
EUVIMED