Simulated data for census-scale entity resolution research without privacy restrictions: a large-scale dataset generated by individual-based modeling
Gates Open Research · 2024
DOI: 10.12688/gatesopenres.15418.2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Gates Open Research, 2024. Verfasst von Beatrix Haddock und 12 weiteren Autor:innen. Beteiligte Einrichtung: Institute for Health Metrics and Evaluation, University of Washington, Seattle, Washington, 98195, USA. Hauptthemen laut MEDLINE: Computer Simulation, Privacy, Censuses. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bill and Melinda Gates Foundation, U.S. Census Bureau.
Abstract (Originalsprache)
Background
Entity resolution (ER) is the process of identifying and linking records that refer to the same real-world entity. ER is a fundamental challenge in data science, and a common barrier to ER research and development is that the data fields used for this fuzzy matching are personally identifiable information, such as name, address, and date of birth. The necessary restrictions on accessing and sharing these authentic data have slowed the work in developing, testing, and adopting new methods and software for ER. We recently released pseudopeople , a Python package that allows users to generate simulated datasets with configurable noise approaching the scale and complexity of the data on which large organizations and federal agencies, like the US Census Bureau regularly perform ER. With pseudopeople, researchers can develop new algorithms and software for ER of US population data without needing access to personal and confidential information.
Methods
We created the simulated population data available for noising with pseudopeople using our Vivarium simulation platform. Our model simulates individuals and their families, households, and employment dynamics over time, which we observe through simulated censuses, surveys, and administrative data collection systems.
Results
Our simulation process produced over 900 gigabytes of simulated censuses, surveys, and administrative data for pseudopeople, representing hundreds of millions of simulants. A sample simulated population of thousands of simulants is now openly available to all users of the pseudopeople package, and large-scale simulated populations of millions and hundreds of millions of simulants are also available by online request through GitHub. These simulated population data are structured for use by the pseudopeople package, which includes additional affordances to add various kinds of noise to the data to provide realistic, sharable challenges for ER researchers.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute for Health Metrics and Evaluation, University of Washington, Seattle, Washington, 98195, USA.
ORCID-Profile: Alix Pletcher, Matt Kappel, Syl Lutze, Abraham Flaxman
Förderung
- Bill and Melinda Gates Foundation
- U.S. Census Bureau
Zitieren
APA
Haddock, B., Pletcher, A., Blair-Stahn, N., Keyes, O., Kappel, M., Bachmeier, S., Lutze, S., Albright, J., Bowman, A., Kinuthia, C., Burke-Conte, Z., Mudambi, R., & Flaxman, A. (2024). Simulated data for census-scale entity resolution research without privacy restrictions: a large-scale dataset generated by individual-based modeling. Gates Open Research. https://doi.org/10.12688/gatesopenres.15418.2
Vancouver
Haddock B, Pletcher A, Blair-Stahn N, Keyes O, Kappel M, Bachmeier S, et al. Simulated data for census-scale entity resolution research without privacy restrictions: a large-scale dataset generated by individual-based modeling. Gates Open Research. 2024. doi:10.12688/gatesopenres.15418.2
BibTeX
@article{Haddock2024,
author = {Haddock, Beatrix and Pletcher, Alix and Blair-Stahn, Nathaniel and Keyes, Os and Kappel, Matt and Bachmeier, Steve and Lutze, Syl and Albright, James and Bowman, Alison and Kinuthia, Caroline and Burke-Conte, Zeb and Mudambi, Rajan and Flaxman, Abraham},
title = {Simulated data for census-scale entity resolution research without privacy restrictions: a large-scale dataset generated by individual-based modeling},
journal = {Gates Open Research},
year = {2024},
doi = {10.12688/gatesopenres.15418.2}
}
EUVIMED