EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Looking for an Even Better Fit? A Personalized Incremental Reinforcement Learning Multimodal Object Referencing Framework

Amr Gomaa, Kiran Gani, Michael Feld, Antonio Krueger

ACM Transactions on Interactive Intelligent Systems · 30.09.2026 · 16 (3) · 1-20

DOI: 10.1145/3820490 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Interactive Intelligent Systems, online erschienen am 17.08.2026. Verfasst von Amr Gomaa und 3 weiteren Autor:innen. Die Arbeit nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: German Ministry of Education and Research (BMBF) under the FedWell project, CAMELOT project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rapid advancement of the automotive industry toward automated and semi-automated vehicles has rendered traditional methods of vehicle interaction, such as touch-based and voice command systems, inadequate for a widening range of non-driving related tasks, such as referencing objects outside of the vehicle. Consequently, research has shifted toward gestural input (e.g., hand, gaze, and head pose gestures) as a more suitable mode of interaction during driving. However, due to the dynamic nature of driving and individual variation, there are significant differences in drivers’ gestural input performance. While, in theory, this inherent variability could be moderated by substantial data-driven machine learning models, prevalent methodologies lean toward constrained, single-instance trained models for object referencing. These models show a limited capacity to continuously adapt to the divergent behaviors of individual drivers and the variety of driving scenarios. To address this, we proposed, in our previous work, an incremental learning approach, adapting to changing user behavior. Although this method proved superior over state-of-the-art approaches, it still lacked in performance and had some computational resource limitations, mainly the small-sized memory needed to store a subset of old data for incremental learning. Therefore, in this work, we propose an alternative solution using reinforcement learning that can be used to overcome memory storage limitations, achieving a zero data storage approach for incremental learning without performance loss. Our newly enhanced solution has been added to our previous framework at https://github.com/amrgomaaelhady/IcRegress .

Förderung

  • German Ministry of Education and Research (BMBF) under the FedWell project
  • CAMELOT project

Zitieren

APA

Gomaa, A., Gani, K., Feld, M., & Krueger, A. (2026). Looking for an Even Better Fit? A Personalized Incremental Reinforcement Learning Multimodal Object Referencing Framework. ACM Transactions on Interactive Intelligent Systems, 16(3), 1-20. https://doi.org/10.1145/3820490

Vancouver

Gomaa A, Gani K, Feld M, Krueger A. Looking for an Even Better Fit? A Personalized Incremental Reinforcement Learning Multimodal Object Referencing Framework. ACM Transactions on Interactive Intelligent Systems. 2026;16(3):1-20. doi:10.1145/3820490

BibTeX
@article{Gomaa2026,
  author = {Gomaa, Amr and Gani, Kiran and Feld, Michael and Krueger, Antonio},
  title = {Looking for an Even Better Fit? A Personalized Incremental Reinforcement Learning Multimodal Object Referencing Framework},
  journal = {ACM Transactions on Interactive Intelligent Systems},
  year = {2026},
  volume = {16},
  number = {3},
  pages = {1--20},
  doi = {10.1145/3820490}
}
Kostenlos starten →