EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting

Youngho Huh iD, Minhwan Noh iD, Dongwoo Ji iD, Yuna Oh iD, Sukkyu Sun

Sensors · 2026

DOI: 10.3390/s26082316 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Youngho Huh und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea. Hauptthemen laut MEDLINE: Arrhythmias, Cardiac, Photoplethysmography. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Seoul RISE Center, Institute for Information & Communications Technology Planning & Evaluation, MOE.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Photoplethysmography (PPG) has emerged as an attractive modality for non-invasive cardiovascular monitoring due to its low cost, unobtrusive nature, and ubiquity in consumer wearable devices. Despite its potential, existing PPG-based arrhythmia detection systems remain limited in scope: (i) most target only atrial fibrillation, (ii) temporal localization of abnormal segments is rarely provided, and (iii) deep learning models lack explainability, hindering adoption in clinical workflows. We present a comprehensive and fully integrated framework for multi-class arrhythmia detection, segmentation, and explainability based on PPG waveforms, Heart Rate Variability (HRV), and structured clinical metadata. The proposed system introduces a CLIP-style contrastive learning module aligning PPG waveforms with clinical variables and rhythm-state textual descriptions using BioBERT; a multitask U-Net architecture performing 4-class classification and 1D segmentation; a Retrieval-Augmented Generation (RAG) pipeline leveraging Gemini Flash large language models to produce guideline-grounded diagnostic reports; and a real-time Streamlit-based web platform supporting inference, visualization, and database storage. The system significantly improves classification accuracy (from 86.27% to 91.19%) and segmentation Dice (from 0.5815 to 0.7167). These results demonstrate the feasibility of a robust, multimodal, and explainable PPG-based arrhythmia monitoring system for real-world applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Arrhythmias, CardiacPhotoplethysmographyHumansLarge Language ModelsSignal Processing, Computer-AssistedHeart RateDeep LearningAlgorithms

Stichwörter der Autor:innen

arrhythmia detectionclinical informationcontrastive learningexplainable AI (XAI)heart rate variability (HRV)large language models (LLMs)photoplethysmography (PPG)retrieval-augmented generation (RAG)

Autor:innen und Institutionen

  • Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea.
  • Division of AI Convergence, Department of Data Science, Dongguk University, Seoul 04620, Republic of Korea.

ORCID-Profile: Youngho Huh, Minhwan Noh, Dongwoo Ji, Yuna Oh

Förderung

  • Seoul RISE Center
  • Institute for Information & Communications Technology Planning & Evaluation
  • MOE
  • MSIT

Zitieren

APA

Huh, Y., Noh, M., Ji, D., Oh, Y., & Sun, S. (2026). Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting. Sensors. https://doi.org/10.3390/s26082316

Vancouver

Huh Y, Noh M, Ji D, Oh Y, Sun S. Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting. Sensors. 2026. doi:10.3390/s26082316

BibTeX
@article{Huh2026,
  author = {Huh, Youngho and Noh, Minhwan and Ji, Dongwoo and Oh, Yuna and Sun, Sukkyu},
  title = {Multimodal PPG-Based Arrhythmia Detection Using a CLIP-Initialized Multi-Task U-Net and LLM-Assisted Reporting},
  journal = {Sensors},
  year = {2026},
  doi = {10.3390/s26082316}
}
Kostenlos starten →