EUVIMEDSkopri EUVIMED
Open access

Targeting COVID-19 and Human Resources for Health News Information Extraction: Algorithm Development and Validation

Mathieu Ravaut iD, Ruochen Zhao iD, Duy Phung iD, Vicky Mengqi Qin iD, Dusan Milovanovic iD, Anita Pienkowska iD, Iva Bojic iD, Josip Car iD, Shafiq Joty iD

JMIR AI · 2024-10-30 · 3 · e55059

DOI: 10.2196/55059 ↗

At a glance

Journal article in JMIR AI, published online on 2024-10-30. Written by Mathieu Ravaut and 8 co-authors. Institution: Nanyang Technological University, Singapore, Singapore. Cited 2 times so far (Crossref); 67 references. The full text is freely available in PubMed Central.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

Global pandemics like COVID-19 put a high amount of strain on health care systems and health workers worldwide. These crises generate a vast amount of news information published online across the globe. This extensive corpus of articles has the potential to provide valuable insights into the nature of ongoing events and guide interventions and policies. However, the sheer volume of information is beyond the capacity of human experts to process and analyze effectively.

Objective

The aim of this study was to explore how natural language processing (NLP) can be leveraged to build a system that allows for quick analysis of a high volume of news articles. Along with this, the objective was to create a workflow comprising human-computer symbiosis to derive valuable insights to support health workforce strategic policy dialogue, advocacy, and decision-making.

Methods

We conducted a review of open-source news coverage from January 2020 to June 2022 on COVID-19 and its impacts on the health workforce from the World Health Organization (WHO) Epidemic Intelligence from Open Sources (EIOS) by synergizing NLP models, including classification and extractive summarization, and human-generated analyses. Our DeepCovid system was trained on 2.8 million news articles in English from more than 3000 internet sources across hundreds of jurisdictions.

Results

Rules-based classification with hand-designed rules narrowed the data set to 8508 articles with high relevancy confirmed in the human-led evaluation. DeepCovid’s automated information targeting component reached a very strong binary classification performance of 98.98 for the area under the receiver operating characteristic curve (ROC-AUC) and 47.21 for the area under the precision recall curve (PR-AUC). Its information extraction component attained good performance in automatic extractive summarization with a mean Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score of 47.76. DeepCovid’s final summaries were used by human experts to write reports on the COVID-19 pandemic.

Conclusions

It is feasible to synergize high-performing NLP models and human-generated analyses to benefit open-source health workforce intelligence. The DeepCovid approach can contribute to an agile and timely global view, providing complementary information to scientific literature.

Subjects

Author keywords

COVID-19NLPSARS-CoV-2classificationdeep learningextractextractionmachine learningmedianatural language processingnewsnews articlessummarizationsummarizesummary

Authors and institutions

  • Nanyang Technological University, Singapore, Singapore.
  • Episteme Systems, Geneva, Switzerland.
  • King's College London, London, United Kingdom.
  • Salesforce Research, San Francisco, CA, United States.

ORCID profiles: Mathieu Ravaut, Ruochen Zhao, Duy Phung, Vicky Mengqi Qin, Dusan Milovanovic, Anita Pienkowska, Iva Bojic, Josip Car, Shafiq Joty

Cite

APA

Ravaut, M., Zhao, R., Phung, D., Qin, V. M., Milovanovic, D., Pienkowska, A., Bojic, I., Car, J., & Joty, S. (2024). Targeting COVID-19 and Human Resources for Health News Information Extraction: Algorithm Development and Validation. JMIR AI, 3, e55059. https://doi.org/10.2196/55059

Vancouver

Ravaut M, Zhao R, Phung D, Qin VM, Milovanovic D, Pienkowska A, et al. Targeting COVID-19 and Human Resources for Health News Information Extraction: Algorithm Development and Validation. JMIR AI. 2024;3:e55059. doi:10.2196/55059

BibTeX
@article{Ravaut2024,
  author = {Ravaut, Mathieu and Zhao, Ruochen and Phung, Duy and Qin, Vicky Mengqi and Milovanovic, Dusan and Pienkowska, Anita and Bojic, Iva and Car, Josip and Joty, Shafiq},
  title = {Targeting COVID-19 and Human Resources for Health News Information Extraction: Algorithm Development and Validation},
  journal = {JMIR AI},
  year = {2024},
  volume = {3},
  pages = {e55059},
  doi = {10.2196/55059}
}
Start for free →