Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting
Sensors · 2026
DOI: 10.3390/s26082316 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Youngho Huh und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea. Hauptthemen laut MEDLINE: Arrhythmias, Cardiac, Photoplethysmography. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Seoul RISE Center, Institute for Information & Communications Technology Planning & Evaluation, MOE.
Abstract (Originalsprache)
Photoplethysmography (PPG) has emerged as an attractive modality for non-invasive cardiovascular monitoring due to its low cost, unobtrusive nature, and ubiquity in consumer wearable devices. Despite its potential, existing PPG-based arrhythmia detection systems remain limited in scope: (i) most target only atrial fibrillation, (ii) temporal localization of abnormal segments is rarely provided, and (iii) deep learning models lack explainability, hindering adoption in clinical workflows. We present a comprehensive and fully integrated framework for multi-class arrhythmia detection, segmentation, and explainability based on PPG waveforms, Heart Rate Variability (HRV), and structured clinical metadata. The proposed system introduces a CLIP-style contrastive learning module aligning PPG waveforms with clinical variables and rhythm-state textual descriptions using BioBERT; a multitask U-Net architecture performing 4-class classification and 1D segmentation; a Retrieval-Augmented Generation (RAG) pipeline leveraging Gemini Flash large language models to produce guideline-grounded diagnostic reports; and a real-time Streamlit-based web platform supporting inference, visualization, and database storage. The system significantly improves classification accuracy (from 86.27% to 91.19%) and segmentation Dice (from 0.5815 to 0.7167). These results demonstrate the feasibility of a robust, multimodal, and explainable PPG-based arrhythmia monitoring system for real-world applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea.
- Division of AI Convergence, Department of Data Science, Dongguk University, Seoul 04620, Republic of Korea.
ORCID-Profile: Youngho Huh, Minhwan Noh, Dongwoo Ji, Yuna Oh
Förderung
- Seoul RISE Center
- Institute for Information & Communications Technology Planning & Evaluation
- MOE
- MSIT
Zitieren
APA
Huh, Y., Noh, M., Ji, D., Oh, Y., & Sun, S. (2026). Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting. Sensors. https://doi.org/10.3390/s26082316
Vancouver
Huh Y, Noh M, Ji D, Oh Y, Sun S. Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting. Sensors. 2026. doi:10.3390/s26082316
BibTeX
@article{Huh2026,
author = {Huh, Youngho and Noh, Minhwan and Ji, Dongwoo and Oh, Yuna and Sun, Sukkyu},
title = {Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting},
journal = {Sensors},
year = {2026},
doi = {10.3390/s26082316}
}
EUVIMED