Original Research
Pulse of the pandemic: Iterative topic filtering for clinical information extraction from social media

https://doi.org/10.1016/j.jbi.2021.103844Get rights and content
Under an Elsevier user license
open archive

Highlights

  • We extract clinically relevant information from COVID-19 Twitter data at scale.

  • Topic modeling and clinical concept extraction are combined to find relevant tweets.

  • A dataset of 1 M COVID-19 related tweets by health-care professionals is analyzed.

  • The proposed method enables real-time trend analysis.

Abstract

The rapid evolution of the COVID-19 pandemic has underscored the need to quickly disseminate the latest clinical knowledge during a public-health emergency. One surprisingly effective platform for healthcare professionals (HCPs) to share knowledge and experiences from the front lines has been social media (for example, the “#medtwitter” community on Twitter). However, identifying clinically-relevant content in social media without manual labeling is a challenge because of the sheer volume of irrelevant data. We present an unsupervised, iterative approach to mine clinically relevant information from social media data, which begins by heuristically filtering for HCP-authored texts and incorporates topic modeling and concept extraction with MetaMap. This approach identifies granular topics and tweets with high clinical relevance from a set of about 52 million COVID-19-related tweets from January to mid-June 2020. We also show that because the technique does not require manual labeling, it can be used to identify emerging topics on a week-to-week basis. Our method can aid in future public-health emergencies by facilitating knowledge transfer among healthcare workers in a rapidly-changing information environment, and by providing an efficient and unsupervised way of highlighting potential areas for clinical research.

Keywords

Data mining
Social media
Information retrieval
Topic modeling
Clinical concept extraction
Public health surveillance

Cited by (0)

1

First three authors contributed equally to this research.