Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime
Sensors · 2025
DOI: 10.3390/s25185831 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Mohammad Afif Kasno und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Artificial Intelligence, College of Advanced Convergence Engineering, Dongguk University, Seoul 04620, Republic of…. Hauptthemen laut MEDLINE: Photoplethysmography, Artificial Intelligence. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and ICT, Ministry of Trade, Industry and Energy, MSIT (Ministry of Science and ICT), Korea.
Abstract (Originalsprache)
This paper presents a real-time smart mirror system combining multiple AI modules for multimodal health monitoring. The proposed platform integrates three core components: facial expression analysis, remote photoplethysmography (rPPG), and conversational AI. A key innovation lies in transforming the Moving Average rPPG (MA-rPPG) model—originally developed for offline batch processing—into a real-time, continuously streaming setup, enabling seamless heart rate and peripheral oxygen saturation (SpO2) monitoring using standard webcams. The system also incorporates the DeepFace facial analysis library for live emotion, age detection, and a Generative Pre-trained Transformer 4o (GPT-4o)-based mental health chatbot with bilingual (English/Korean) support and voice synthesis. Embedded into a touchscreen mirror with Graphical User Interface (GUI), this solution delivers ambient, low-interruption interaction and real-time user feedback. By unifying these AI modules within an interactive smart mirror, our findings demonstrate the feasibility of integrating multimodal sensing (rPPG, affect detection) and conversational AI into a real-time smart mirror platform. This system is presented as a feasibility-stage prototype to promote real-time health awareness and empathetic feedback. The physiological validation was limited to a single subject, and the user evaluation constituted only a small formative assessment; therefore, results should be interpreted strictly as preliminary feasibility evidence. The system is not intended to provide clinical diagnosis or generalizable accuracy at this stage.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science and Artificial Intelligence, College of Advanced Convergence Engineering, Dongguk University, Seoul 04620, Republic of Korea.
- Faculty of Electrical Technology and Engineering, Universiti Teknikal Malaysia Melaka, Melaka 76100, Malaysia.
ORCID-Profile: Mohammad Afif Kasno
Förderung
- Ministry of Science and ICT
- Ministry of Trade, Industry and Energy
- MSIT (Ministry of Science and ICT), Korea
- ITRC (Information Technology Research Center)
- IITP (Institute for Information & Communications Technology Planning & Evaluation)
- Commercialization Promotion Agency for R&D Outcomes (COMPA)
Zitieren
APA
Kasno, M. A., & Jung, J. W. (2025). Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime. Sensors. https://doi.org/10.3390/s25185831
Vancouver
Kasno MA, Jung JW. Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime. Sensors. 2025. doi:10.3390/s25185831
BibTeX
@article{Kasno2025,
author = {Kasno, Mohammad Afif and Jung, Jin-Woo},
title = {Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime},
journal = {Sensors},
year = {2025},
doi = {10.3390/s25185831}
}
EUVIMED