Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit
Sensors · 2026
DOI: 10.3390/s26031060 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Céline Madeleine Aldenhoven und 6 weiteren Autor:innen. Beteiligte Einrichtung: Institute for Digital Medicine, University Hospital Bonn, University of Bonn, 53113 Bonn, Germany. Hauptthemen laut MEDLINE: Emotions, Smartphone, Pattern Recognition, Automated. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: German Federal Ministry of Research, Technology and Space.
Abstract (Originalsprache)
Facial features hold information about a person’s emotions, motor function, or genetic defects. Since most current mobile devices are capable of real-time face detection using cameras and depth sensors, real-time facial analysis can be utilized in several mobile use cases. Understanding the real-time emotion recognition capabilities of device sensors and frameworks is vital for developing new, valid applications. Therefore, we evaluated on-device emotion recognition using Apple’s ARKit on an iPhone 14 Pro. A native app elicited 36 blend shape-specific movements and 7 discrete emotions from N=31 healthy adults. Per frame, standardized ARKit blend shapes were classified using a prototype-based cosine similarity metric; performance was summarized as accuracy and area under the receiver operating characteristic curves. Cosine similarity achieved an overall accuracy of 68.3%, exceeding the mean of three human raters (58.9%; +9.4 percentage points, ≈16% relative). Per-emotion accuracy was highest for joy, fear, sadness, and surprise, and competitive for anger, disgust, and contempt. AUCs were ≥0.84 for all classes. The method runs in real time on-device using only vector operations, preserving privacy and minimizing compute. These results indicate that a simple, interpretable cosine-similarity classifier over ARKit blend shapes delivers human-comparable, real-time facial emotion recognition on commodity hardware, supporting privacy-preserving mobile applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute for Digital Medicine, University Hospital Bonn, University of Bonn, 53113 Bonn, Germany.
- German Center for Neurodegenerative Diseases (DZNE), 53127 Bonn, Germany.
- Department of Old Age Psychiatry and Cognitive Disorders, University Hospital Bonn, University of Bonn, 53127 Bonn, Germany.
ORCID-Profile: Céline Madeleine Aldenhoven, Leon Nissen, Marie Heinemann, Cem Doğdu, Stephan Jonas, Lara Marie Reimer
Förderung
- German Federal Ministry of Research, Technology and Space
Zitieren
APA
Aldenhoven, C. M., Nissen, L., Heinemann, M., Doğdu, C., Hanke, A., Jonas, S., & Reimer, L. M. (2026). Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit. Sensors. https://doi.org/10.3390/s26031060
Vancouver
Aldenhoven CM, Nissen L, Heinemann M, Doğdu C, Hanke A, Jonas S, et al. Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit. Sensors. 2026. doi:10.3390/s26031060
BibTeX
@article{Aldenhoven2026,
author = {Aldenhoven, Céline Madeleine and Nissen, Leon and Heinemann, Marie and Doğdu, Cem and Hanke, Alexander and Jonas, Stephan and Reimer, Lara Marie},
title = {Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple’s ARKit},
journal = {Sensors},
year = {2026},
doi = {10.3390/s26031060}
}
EUVIMED