EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Gaze-Swin: Enhancing Gaze Estimation with a Hybrid CNN-Transformer Network and Dropkey Mechanism

Ruijie Zhao, Yuhuan Wang, Sihui Luo, Suyao Shou, Pinyan Tang

Electronics · 12.01.2024 · 13 (2) · 328

DOI: 10.3390/electronics13020328 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 12.01.2024. Verfasst von Ruijie Zhao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Zhejiang Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Gaze estimation, which seeks to reveal where a person is looking, provides a crucial clue for understanding human intentions and behaviors. Recently, Visual Transformer has achieved promising results in gaze estimation. However, dividing facial images into patches compromises the integrity of the image structure, which limits the inference performance. To tackle this challenge, we present Gaze-Swin, an end-to-end gaze estimation model formed with a dual-branch CNN-Transformer architecture. In Gaze-Swin, we adopt the Swin Transformer as the backbone network due to its effectiveness in handling long-range dependencies and extracting global features. Additionally, we incorporate a convolutional neural network as an auxiliary branch to capture local facial features and intricate texture details. To further enhance robustness and address overfitting issues in gaze estimation, we replace the original self-attention in the Transformer branch with Dropkey Assisted Attention (DA-Attention). In particular, this DA-Attention treats keys in the Transformer block as Dropout units and employs a decay Dropout rate schedule to preserve crucial gaze representations in deeper layers. Comprehensive experiments on three benchmark datasets demonstrate the superior performance of our method in comparison to the state of the art.

Förderung

  • Natural Science Foundation of Zhejiang Province

Zitieren

APA

Zhao, R., Wang, Y., Luo, S., Shou, S., & Tang, P. (2024). Gaze-Swin: Enhancing Gaze Estimation with a Hybrid CNN-Transformer Network and Dropkey Mechanism. Electronics, 13(2), 328. https://doi.org/10.3390/electronics13020328

Vancouver

Zhao R, Wang Y, Luo S, Shou S, Tang P. Gaze-Swin: Enhancing Gaze Estimation with a Hybrid CNN-Transformer Network and Dropkey Mechanism. Electronics. 2024;13(2):328. doi:10.3390/electronics13020328

BibTeX
@article{Zhao2024,
  author = {Zhao, Ruijie and Wang, Yuhuan and Luo, Sihui and Shou, Suyao and Tang, Pinyan},
  title = {Gaze-Swin: Enhancing Gaze Estimation with a Hybrid CNN-Transformer Network and Dropkey Mechanism},
  journal = {Electronics},
  year = {2024},
  volume = {13},
  number = {2},
  pages = {328},
  doi = {10.3390/electronics13020328}
}
Kostenlos starten →