EUVIMEDObjevte EUVIMED
Open access · CC BY 4.0

Experience Filter: Translating Heterogeneous Signals into Motion

Ekrem Yüksel iD, Alihan Bakir iD, Jamie Paik iD

Advanced Science · 2026-11-10

DOI: 10.1002/advs.78602 ↗

At a glance

Journal article in Advanced Science, published online on 2026-11-10. Written by Ekrem Yüksel and 2 co-authors. Institution: Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland. 43 references. The full text is freely available (open access, CC BY 4.0). Funded by: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Human experience is inherently rich and multisensory, composed of intertwined visual, auditory, and kinesthetic feedback. Digital media capture only its audiovisual dimensions, omitting its mechanosensory dimensions. While full‐body (kinesthetic) haptic feedback can enrich media, no general method exists for deriving structured motion from heterogeneous signals and combining it with the audiovisual content for rendering. The Experience Filter maps multimedia signals into structured motion trajectories through a common pipeline of modality‐specific feature extraction followed by a modality‐agnostic shared encoding, recombining them into a synchronized multisensory experience: each signal is reduced to a shared descriptor space – signal intensity, dominant spectral feature, and distance from neutral axis – then mapped to a trajectory via a fixed, deterministic, training‐free encoding. Because this encoding space is hardware‐independent, the motion can be retargeted to robotic platforms of arbitrary degrees of freedom and morphology. A feature‐sensitivity analysis shows that the descriptor axes are independent and the encoding‐space topology follows from the chosen descriptors rather than being fixed. The framework is demonstrated on real‐world music and images, producing continuous, synchronized image‐audio‐motion experiences without manual authoring. The Experience Filter thus provides an analytical mapping from multimedia signals to combined image‐audio‐motion rendering, enabling expressive, user‐adjustable experience across robotic platforms.

Subjects

Author keywords

haptic feedbackhuman‐robot interactionmultisensorial feedbacksignal‐to‐motion encodingtactile force feedback

Authors and institutions

  • Ecole Polytechnique Fédérale de Lausanne (EPFL), Reconfigurable Robotics Laboratory, Lausanne, Switzerland.

ORCID profiles: Ekrem Yüksel, Alihan Bakir, Jamie Paik

Funding

  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung

Cite

APA

Yüksel, E., Bakir, A., & Paik, J. (2026). Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. https://doi.org/10.1002/advs.78602

Vancouver

Yüksel E, Bakir A, Paik J. Experience Filter: Translating Heterogeneous Signals into Motion. Advanced Science. 2026. doi:10.1002/advs.78602

BibTeX
@article{Yksel2026,
  author = {Yüksel, Ekrem and Bakir, Alihan and Paik, Jamie},
  title = {Experience Filter: Translating Heterogeneous Signals into Motion},
  journal = {Advanced Science},
  year = {2026},
  doi = {10.1002/advs.78602}
}
Start for free →