E-Gaze: Gaze Estimation With Event Camera
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-07 · 46 (7) · 4796-4811
DOI: 10.1109/tpami.2024.3359606 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-07. Verfasst von Nealson Li und 2 weiteren Autor:innen. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 54 Literaturangaben. Gefördert von: CoCoSys, Semiconductor Research Corporation, Defense Advanced Research Projects Agency.
Abstract (Originalsprache)
Near-eye gaze estimation is a task that maps the recording of an eye captured by an adjacent camera to the direction of a person's gaze in space. In contrast to frame-based cameras, event cameras are characterized by high sensing rates, low latency, sparse asynchronous data outputs, and high dynamic range, which are well suited for recording the fast eye movements. However, algorithms and system designs that operate on frame-based cameras are not applicable to event-based data, due to the natural differences in the data characteristics. In this work, we study the pattern of near-eye event-based data streams and extract eye features to estimate gaze. First, by analyzing eye parts and movements, and harnessing the polar, spatial, and temporal distribution of the events, we introduce a real-time pipeline to extract pupil features. Second, we present a recurrent neural network with a proposed coordinate-to-angle loss function to accurately estimate gaze from pupil feature sequence. We demonstrated that our system achieves accurate real-time estimation with angular accuracy of 0.46 ° and update rates of 950 Hz, thus opening up avenues for novel applications. To our knowledge, this is the first system that operates only on event-based data to perform gaze estimation.
Förderung
- CoCoSys
- Semiconductor Research Corporation
- Defense Advanced Research Projects Agency
Zitieren
APA
Li, N., Chang, M., & Raychowdhury, A. (2024). E-Gaze: Gaze Estimation With Event Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(7), 4796-4811. https://doi.org/10.1109/tpami.2024.3359606
Vancouver
Li N, Chang M, Raychowdhury A. E-Gaze: Gaze Estimation With Event Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(7):4796-4811. doi:10.1109/tpami.2024.3359606
BibTeX
@article{Li2024,
author = {Li, Nealson and Chang, Muya and Raychowdhury, Arijit},
title = {E-Gaze: Gaze Estimation With Event Camera},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {7},
pages = {4796--4811},
doi = {10.1109/tpami.2024.3359606}
}
EUVIMED