EUVIMEDEUVIMED entdecken
Crossref

Efficient data integration under prior probability shift

Ming-Yueh Huang iD, Jing Qin iD, Chiung-Yu Huang iD

Biometrics · 27.03.2024 · 80 (2)

DOI: 10.1093/biomtc/ujae035 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biometrics, online erschienen am 20.05.2024. Verfasst von Ming-Yueh Huang und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Statistical Science, Academia Sinica, Taipei 11529, Taiwan. Hauptthemen laut MEDLINE: Algorithms, Probability, Computer Simulation, Models, Statistical. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 21 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Conventional supervised learning usually operates under the premise that data are collected from the same underlying population. However, challenges may arise when integrating new data from different populations, resulting in a phenomenon known as dataset shift. This paper focuses on prior probability shift, where the distribution of the outcome varies across datasets but the conditional distribution of features given the outcome remains the same. To tackle the challenges posed by such shift, we propose an estimation algorithm that can efficiently combine information from multiple sources. Unlike existing methods that are restricted to discrete outcomes, the proposed approach accommodates both discrete and continuous outcomes. It also handles high-dimensional covariate vectors through variable selection using an adaptive least absolute shrinkage and selection operator penalty, producing efficient estimates that possess the oracle property. Moreover, a novel semiparametric likelihood ratio test is proposed to check the validity of prior probability shift assumptions by embedding the null conditional density function into Neyman’s smooth alternatives (Neyman, 1937) and testing study-specific parameters. We demonstrate the effectiveness of our proposed method through extensive simulations and a real data example. The proposed methods serve as a useful addition to the repertoire of tools for dealing dataset shifts.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AlgorithmsProbabilityComputer SimulationModels, StatisticalHumansLikelihood FunctionsBiometryData Interpretation, StatisticalSupervised Machine Learning

Stichwörter der Autor:innen

dataset shiftpenalized likelihoodprofile likelihoodsemiparametric efficiency

Autor:innen und Institutionen

  • Institute of Statistical Science, Academia Sinica, Taipei 11529, Taiwan.
  • Biostatistics Research Branch, National Institute of Allergy and Infectious Diseases, National Institutes of Health,Bethesda, MD 20892, United States.
  • Department of Epidemiology and Biostatistics, University of California, San Francisco, CA 94158, United States.

ORCID-Profile: Ming-Yueh Huang, Jing Qin, Chiung-Yu Huang

Zitieren

APA

Huang, M. Y., Qin, J., & Huang, C. Y. (2024). Efficient data integration under prior probability shift. Biometrics, 80(2). https://doi.org/10.1093/biomtc/ujae035

Vancouver

Huang MY, Qin J, Huang CY. Efficient data integration under prior probability shift. Biometrics. 2024;80(2). doi:10.1093/biomtc/ujae035

BibTeX
@article{Huang2024,
  author = {Huang, Ming-Yueh and Qin, Jing and Huang, Chiung-Yu},
  title = {Efficient data integration under prior probability shift},
  journal = {Biometrics},
  year = {2024},
  volume = {80},
  number = {2},
  doi = {10.1093/biomtc/ujae035}
}
Kostenlos starten →