EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit

Céline Madeleine Aldenhoven iD, Leon Nissen iD, Marie Heinemann iD, Cem Doğdu iD, Alexander Hanke, Stephan Jonas iD, Lara Marie Reimer iD

Sensors · 2026

DOI: 10.3390/s26031060 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Céline Madeleine Aldenhoven und 6 weiteren Autor:innen. Beteiligte Einrichtung: Institute for Digital Medicine, University Hospital Bonn, University of Bonn, 53113 Bonn, Germany. Hauptthemen laut MEDLINE: Emotions, Smartphone, Pattern Recognition, Automated. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: German Federal Ministry of Research, Technology and Space.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Facial features hold information about a person’s emotions, motor function, or genetic defects. Since most current mobile devices are capable of real-time face detection using cameras and depth sensors, real-time facial analysis can be utilized in several mobile use cases. Understanding the real-time emotion recognition capabilities of device sensors and frameworks is vital for developing new, valid applications. Therefore, we evaluated on-device emotion recognition using Apple’s ARKit on an iPhone 14 Pro. A native app elicited 36 blend shape-specific movements and 7 discrete emotions from N=31 healthy adults. Per frame, standardized ARKit blend shapes were classified using a prototype-based cosine similarity metric; performance was summarized as accuracy and area under the receiver operating characteristic curves. Cosine similarity achieved an overall accuracy of 68.3%, exceeding the mean of three human raters (58.9%; +9.4 percentage points, ≈16% relative). Per-emotion accuracy was highest for joy, fear, sadness, and surprise, and competitive for anger, disgust, and contempt. AUCs were ≥0.84 for all classes. The method runs in real time on-device using only vector operations, preserving privacy and minimizing compute. These results indicate that a simple, interpretable cosine-similarity classifier over ARKit blend shapes delivers human-comparable, real-time facial emotion recognition on commodity hardware, supporting privacy-preserving mobile applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

EmotionsSmartphonePattern Recognition, AutomatedHumansFacial ExpressionMobile ApplicationsAdultFace

Stichwörter der Autor:innen

ARKitemotion recognitionface trackingreal-timesensors

Autor:innen und Institutionen

  • Institute for Digital Medicine, University Hospital Bonn, University of Bonn, 53113 Bonn, Germany.
  • German Center for Neurodegenerative Diseases (DZNE), 53127 Bonn, Germany.
  • Department of Old Age Psychiatry and Cognitive Disorders, University Hospital Bonn, University of Bonn, 53127 Bonn, Germany.

ORCID-Profile: Céline Madeleine Aldenhoven, Leon Nissen, Marie Heinemann, Cem Doğdu, Stephan Jonas, Lara Marie Reimer

Förderung

  • German Federal Ministry of Research, Technology and Space

Zitieren

APA

Aldenhoven, C. M., Nissen, L., Heinemann, M., Doğdu, C., Hanke, A., Jonas, S., & Reimer, L. M. (2026). Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit. Sensors. https://doi.org/10.3390/s26031060

Vancouver

Aldenhoven CM, Nissen L, Heinemann M, Doğdu C, Hanke A, Jonas S, et al. Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit. Sensors. 2026. doi:10.3390/s26031060

BibTeX
@article{Aldenhoven2026,
  author = {Aldenhoven, Céline Madeleine and Nissen, Leon and Heinemann, Marie and Doğdu, Cem and Hanke, Alexander and Jonas, Stephan and Reimer, Lara Marie},
  title = {Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit},
  journal = {Sensors},
  year = {2026},
  doi = {10.3390/s26031060}
}
Kostenlos starten →