EUVIMEDEUVIMED entdecken
Open Access

Generating Contextual Variables From Web-Based Data for Health Research: Tutorial on Web Scraping, Text Mining, and Spatial Overlay Analysis

Pablo Galvez-Hernandez iD, Angelina Gonzalez-Viana iD, Luis Gonzalez-de Paz iD, Ketan Shankardass iD, Carles Muntaner iD

JMIR Public Health and Surveillance · 08.01.2024 · 10 · e50379

DOI: 10.2196/50379 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Public Health and Surveillance, online erschienen am 08.01.2024. Verfasst von Pablo Galvez-Hernandez und 4 weiteren Autor:innen. Beteiligte Einrichtung: Lawrence S Bloomberg Faculty of Nursing, University of Toronto, Toronto, ON, Canada. Hauptthemen laut MEDLINE: Data Mining, Algorithms. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Contextual variables that capture the characteristics of delimited geographic or jurisdictional areas are vital for health and social research. However, obtaining data sets with contextual-level data can be challenging in the absence of monitoring systems or public census data.

Objective

We describe and implement an 8-step method that combines web scraping, text mining, and spatial overlay analysis (WeTMS) to transform extensive text data from government websites into analyzable data sets containing contextual data for jurisdictional areas.

Methods

This tutorial describes the method and provides resources for its application by health and social researchers. We used this method to create data sets of health assets aimed at enhancing older adults’ social connections (eg, activities and resources such as walking groups and senior clubs) across the 374 health jurisdictions in Catalonia from 2015 to 2022. These assets are registered on a web-based government platform by local stakeholders from various health and nonhealth organizations as part of a national public health program. Steps 1 to 3 involved defining the variables of interest, identifying data sources, and using Python to extract information from 50,000 websites linked to the platform. Steps 4 to 6 comprised preprocessing the scraped text, defining new variables to classify health assets based on social connection constructs, analyzing word frequencies in titles and descriptions of the assets, creating topic-specific dictionaries, implementing a rule-based classifier in R, and verifying the results. Steps 7 and 8 integrate the spatial overlay analysis to determine the geographic location of each asset. We conducted a descriptive analysis of the data sets to report the characteristics of the assets identified and the patterns of asset registrations across areas.

Results

We identified and extracted data from 17,305 websites describing health assets. The titles and descriptions of the activities and resources contained 12,560 and 7301 unique words, respectively. After applying our classifier and spatial analysis algorithm, we generated 2 data sets containing 9546 health assets (5022 activities and 4524 resources) with the potential to enhance social connections among older adults. Stakeholders from 318 health jurisdictions registered identified assets on the platform between July 2015 and December 2022. The agreement rate between the classification algorithm and verified data sets ranged from 62.02% to 99.47% across variables. Leisure and skill development activities were the most prevalent (1844/5022, 36.72%). Leisure and cultural associations, such as social clubs for older adults, were the most common resources (878/4524, 19.41%). Health asset registration varied across areas, ranging between 0 and 263 activities and 0 and 265 resources.

Conclusions

The sequential use of WeTMS offers a robust method for generating data sets containing contextual-level variables from internet text data. This study can guide health and social researchers in efficiently generating ready-to-analyze data sets containing contextual variables.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Data MiningAlgorithmsHumansAgedGovernmentInternetSpatial Analysis

Stichwörter der Autor:innen

contextual variableshealth assetshealth services researchmultilevel analysisprogram evaluationsocial connectionsocial environmentspatial overlay analysistext miningweb scraping

Autor:innen und Institutionen

  • Lawrence S Bloomberg Faculty of Nursing, University of Toronto, Toronto, ON, Canada.
  • Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada.
  • Public Health Agency of Catalonia, Health Department, Barcelona, Spain.
  • Primary Healthcare Transversal Research Group, Institut d'Investigacions Biomèdiques August Pi i Sunyer, Barcelona, Spain.
  • Consorci d'Atenció Primària de Salut Barcelona Esquerra, Barcelona, Spain.
  • Department of Heath Sciences, Wilfrid Laurier University, Waterloo, ON, Canada.

ORCID-Profile: Pablo Galvez-Hernandez, Angelina Gonzalez-Viana, Luis Gonzalez-de Paz, Ketan Shankardass, Carles Muntaner

Zitieren

APA

Galvez-Hernandez, P., Gonzalez-Viana, A., Gonzalez-de Paz, L., Shankardass, K., & Muntaner, C. (2024). Generating Contextual Variables From Web-Based Data for Health Research: Tutorial on Web Scraping, Text Mining, and Spatial Overlay Analysis. JMIR Public Health and Surveillance, 10, e50379. https://doi.org/10.2196/50379

Vancouver

Galvez-Hernandez P, Gonzalez-Viana A, Gonzalez-de Paz L, Shankardass K, Muntaner C. Generating Contextual Variables From Web-Based Data for Health Research: Tutorial on Web Scraping, Text Mining, and Spatial Overlay Analysis. JMIR Public Health and Surveillance. 2024;10:e50379. doi:10.2196/50379

BibTeX
@article{GalvezHernandez2024,
  author = {Galvez-Hernandez, Pablo and Gonzalez-Viana, Angelina and Gonzalez-de Paz, Luis and Shankardass, Ketan and Muntaner, Carles},
  title = {Generating Contextual Variables From Web-Based Data for Health Research: Tutorial on Web Scraping, Text Mining, and Spatial Overlay Analysis},
  journal = {JMIR Public Health and Surveillance},
  year = {2024},
  volume = {10},
  pages = {e50379},
  doi = {10.2196/50379}
}
Kostenlos starten →