EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime

Mohammad Afif Kasno iD, Jin-Woo Jung

Sensors · 2025

DOI: 10.3390/s25185831 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Mohammad Afif Kasno und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Artificial Intelligence, College of Advanced Convergence Engineering, Dongguk University, Seoul 04620, Republic of…. Hauptthemen laut MEDLINE: Photoplethysmography, Artificial Intelligence. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and ICT, Ministry of Trade, Industry and Energy, MSIT (Ministry of Science and ICT), Korea.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper presents a real-time smart mirror system combining multiple AI modules for multimodal health monitoring. The proposed platform integrates three core components: facial expression analysis, remote photoplethysmography (rPPG), and conversational AI. A key innovation lies in transforming the Moving Average rPPG (MA-rPPG) model—originally developed for offline batch processing—into a real-time, continuously streaming setup, enabling seamless heart rate and peripheral oxygen saturation (SpO2) monitoring using standard webcams. The system also incorporates the DeepFace facial analysis library for live emotion, age detection, and a Generative Pre-trained Transformer 4o (GPT-4o)-based mental health chatbot with bilingual (English/Korean) support and voice synthesis. Embedded into a touchscreen mirror with Graphical User Interface (GUI), this solution delivers ambient, low-interruption interaction and real-time user feedback. By unifying these AI modules within an interactive smart mirror, our findings demonstrate the feasibility of integrating multimodal sensing (rPPG, affect detection) and conversational AI into a real-time smart mirror platform. This system is presented as a feasibility-stage prototype to promote real-time health awareness and empathetic feedback. The physiological validation was limited to a single subject, and the user evaluation constituted only a small formative assessment; therefore, results should be interpreted strictly as preliminary feasibility evidence. The system is not intended to provide clinical diagnosis or generalizable accuracy at this stage.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PhotoplethysmographyArtificial IntelligenceHumansHeart RateMonitoring, PhysiologicFacial ExpressionAdultFeasibility StudiesFemaleMaleUser-Computer Interface

Stichwörter der Autor:innen

ambient assisted living (AAL)facial emotion recognitionhuman-centered AImental health chatbotreal-time health monitoringremote photoplethysmography (rPPG)smart mirror

Autor:innen und Institutionen

  • Department of Computer Science and Artificial Intelligence, College of Advanced Convergence Engineering, Dongguk University, Seoul 04620, Republic of Korea.
  • Faculty of Electrical Technology and Engineering, Universiti Teknikal Malaysia Melaka, Melaka 76100, Malaysia.

ORCID-Profile: Mohammad Afif Kasno

Förderung

  • Ministry of Science and ICT
  • Ministry of Trade, Industry and Energy
  • MSIT (Ministry of Science and ICT), Korea
  • ITRC (Information Technology Research Center)
  • IITP (Institute for Information & Communications Technology Planning & Evaluation)
  • Commercialization Promotion Agency for R&D Outcomes (COMPA)

Zitieren

APA

Kasno, M. A., & Jung, J. W. (2025). Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime. Sensors. https://doi.org/10.3390/s25185831

Vancouver

Kasno MA, Jung JW. Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime. Sensors. 2025. doi:10.3390/s25185831

BibTeX
@article{Kasno2025,
  author = {Kasno, Mohammad Afif and Jung, Jin-Woo},
  title = {Feasibility of an AI-Enabled Smart Mirror Integrating MA-rPPG, Facial Affect, and Conversational Guidance in Realtime},
  journal = {Sensors},
  year = {2025},
  doi = {10.3390/s25185831}
}
Kostenlos starten →