EUVIMEDEUVIMED entdecken
Crossref

Panic Prediction from Digital Phenotyping: Subject-Level Cross-Validation Reveals Limited Between-Person Generalization

Minoru Hattori iD, Naoko Hasunuma

Methods of Information in Medicine · 01.09.2025 · 64 (03/06) · 129-133

DOI: 10.1055/a-2895-5222 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Methods of Information in Medicine, erschienen am 01.09.2025. Verfasst von Minoru Hattori und 1 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Biomedical and Health Sciences, Hiroshima University, Department of Medical Education, Japan, Hiroshima. Hauptthemen laut MEDLINE: Phenotype.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

A publicly shared digital phenotyping dataset was recently used to train models predicting day-before-panic (DBP) events, reporting a cross-validated receiver operating characteristic area under the curve (ROC-AUC) of around 0.90. Because the stratified five-fold cross-validation (CV) allowed the same participant's days in both training and test folds, it was unclear whether this reflects between-person generalization or information leakage. We reanalyzed the public dataset to quantify this distinction.

Methods

We reanalyzed the public dataset (3,969 person-days; 254 DBP events, 6.4%, across 19 of 43 participants) with the same XGBoost classifier and 71 features, comparing stratified five-fold cross-validation with leave-one-subject-out (LOSO) cross-validation. A retrospective diagnostic decomposition was applied within-subject z-scoring to dynamic features before LOSO.

Results

We reproduced the original ROC-AUC at 0.895 ± 0.011. Under LOSO, pooled ROC-AUC dropped to 0.489 (bootstrap 95% CI: 0.325–0.643, including 0.50). The SHapley Additive exPlanations (SHAP) ranking under stratified CV was dominated by static traits (CTQ subscales, BRIAN, SPAQ). The diagnostic decomposition yielded LOSO ROC-AUC 0.516 (precision–recall area under the curve [PR-AUC] 0.114), suggesting a weak residual within-person signal, though this should not be interpreted as prospective performance.

Conclusion

Robust between-person generalization was not confirmed under subject-held-out evaluation. The feature ranking was trait-dominated, though the data cannot definitively distinguish identity discrimination from other between-person variance sources. Digital phenotyping studies should report subject-level cross-validation alongside standard metrics, per the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) and Prediction model Risk of Bias Assessment Tool (PROBAST) guidelines.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PhenotypeHumansROC CurvePrediction AlgorithmsArea Under Curve

Autor:innen und Institutionen

  • Graduate School of Biomedical and Health Sciences, Hiroshima University, Department of Medical Education, Japan, Hiroshima.

ORCID-Profile: Minoru Hattori

Zitieren

APA

Hattori, M., & Hasunuma, N. (2025). Panic Prediction from Digital Phenotyping: Subject-Level Cross-Validation Reveals Limited Between-Person Generalization. Methods of Information in Medicine, 64(03/06), 129-133. https://doi.org/10.1055/a-2895-5222

Vancouver

Hattori M, Hasunuma N. Panic Prediction from Digital Phenotyping: Subject-Level Cross-Validation Reveals Limited Between-Person Generalization. Methods of Information in Medicine. 2025;64(03/06):129-133. doi:10.1055/a-2895-5222

BibTeX
@article{Hattori2025,
  author = {Hattori, Minoru and Hasunuma, Naoko},
  title = {Panic Prediction from Digital Phenotyping: Subject-Level Cross-Validation Reveals Limited Between-Person Generalization},
  journal = {Methods of Information in Medicine},
  year = {2025},
  volume = {64},
  number = {03/06},
  pages = {129--133},
  doi = {10.1055/a-2895-5222}
}
Kostenlos starten →