Figures
Abstract
During the COVID-19 pandemic, US populations have experienced elevated rates of financial and psychological distress that could lead to increases in suicide rates. Rapid ongoing mental health monitoring is critical for early intervention, especially in regions most affected by the pandemic, yet traditional surveillance data are available only after long lags. Novel information on real-time population isolation and concerns stemming from the pandemic’s social and economic impacts, via cellular mobility tracking and online search data, are potentially important interim surveillance resources. Using these measures, we employed transfer function model time-series analyses to estimate associations between daily mobility indicators (proportion of cellular devices completely at home and time spent at home) and Google Health Trends search volumes for terms pertaining to economic stress, mental health, and suicide during 2020 and 2021 both nationally and in New York City. During the first pandemic wave in early-spring 2020, over 50% of devices remained completely at home and searches for economic stressors exceeded 60,000 per 10 million. We found large concurrent associations across analyses between declining mobility and increasing searches for economic stressor terms (national proportion of devices at home: cross-correlation coefficient (CC) = 0.6 (p-value <0.001)). Nationally, we also found strong associations between declining mobility and increasing mental health and suicide-related searches (time at home: mood/anxiety CC = 0.53 (<0.001), social stressor CC = 0.51 (<0.001), suicide seeking CC = 0.37 (0.006)). Our findings suggest that pandemic-related isolation coincided with acute economic distress and may be a risk factor for poor mental health and suicidal behavior. These emergent relationships warrant ongoing attention and causal assessment given the potential for long-term psychological impact and suicide death. As US populations continue to face stress, Google search data can be used to identify possible warning signs from real-time changes in distributions of population thought patterns.
Citation: Gimbrone C, Rutherford C, Kandula S, Martínez-Alés G, Shaman J, Olfson M, et al. (2021) Associations between COVID-19 mobility restrictions and economic, mental health, and suicide-related concerns in the US using cellular phone GPS and Google search volume data. PLoS ONE 16(12): e0260931. https://doi.org/10.1371/journal.pone.0260931
Editor: Sinan Kardeş, Istanbul University Istanbul Faculty of Medicine: Istanbul Universitesi Istanbul Tip Fakultesi, TURKEY
Received: July 13, 2021; Accepted: November 19, 2021; Published: December 22, 2021
Copyright: © 2021 Gimbrone et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant geographically aggregated mobility data analyzed within this manuscript are located in the Supporting Information files. Disaggregate mobility data are available from SafeGraph (contact for SafeGraph Data for Academics Program via https://urldefense.proofpoint.com/v2/url?u=https-3A__www.safegraph.com_academics&d=DwIGaQ&c=G2MiLlal7SXE3PeSnG8W6_JBU6FcdVjSsBSbw6gcR0U&r=RSUMhliArNvjr4JmfG5vN4-kWnvPJ37K953kq38kqpw&m=4n2qQ28Ga7KPxRAaVoo9cmxu2O7hybiFBNCUdDdYKTY&s=3m42NmVAERHJ9sEYJEUg-fFydwEaofOSU27zzHlKeRE&e=). Search volume data cannot be shared publicly because they are owned by third parties and the authors do not have permission to share the data. Search volume data are available from Google Health Trends API (Google Health Trends API access may be requested by filling out this form https://docs.google.com/forms/d/e/1FAIpQLSenHdGiGl1YF-7rVDDmmulN8R-ra9MnGLLs7gIIaAX9VHPdPg/viewform or emailing trends-api-support@google.com).
Funding: This work was funded by the National Institute of Mental Health (https://www.nimh.nih.gov/): R01-MH121410, Co-PI’s: KK,JS. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: Jeffrey Shaman and Columbia University declare partial ownership of SK Analytics. Sasikiran Kandula was a consultant for SK Analytics. Jeffrey Shaman was a consultant for BNI. Katherine Keyes has been compensated as an expert witness in litigation. No other disclosures were reported. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
The SARS-CoV-2 pandemic and resulting illness and death related to COVID-19 are associated with deleterious effects on population mental health in the US [1–8]. There are several potential mechanisms through which the pandemic may affect mental health and suicide risk, including loss and bereavement of loved ones [9, 10], social isolation and loneliness due to quarantine and public health recommendations limiting mobility [11–13], diminished access to mental healthcare and treatment within an increasingly overwhelmed medical system [14–16], reduced ability of existing safety nets to mitigate abuse [17, 18], and unemployment and financial instability due to restrictions on business activity [19, 20]. Available data indicate increased rates of psychiatric disorders, such as depression [2, 7, 8], and exacerbated existing mental health disorders among COVID-19 patients in the first waves of the pandemic [21, 22]. Contrary to many hypotheses, however, provisional death data suggest that suicide mortality in the US did not increase during the initial pandemic outbreak [23, 24], yet there is widespread concern that rising financial insecurity and deteriorating mental health could lead to subsequent increases [25–29]. Increased suicide rates have been observed after previous pandemics [30] and recessions [31], and in other areas of the world during 2020 [32]. Using contemporary information to monitor the dynamic medium- and long-term impacts of the pandemic on economic, mental health, and suicide-related concerns is critical to guiding prevention efforts.
Accurate assessment of the evolving effects of the pandemic on economic stress, mental health, and suicide risk requires a fluid measure of population behavior over time, such as mobility (e.g. leaving one’s home or neighborhood). While cities and states passed stay-at-home orders on specific dates that can be tracked, the psychological impacts of these policies remain unclear [13] and mobility declines often preceded their passage [33, 34]. The politicization of the pandemic also spurred divisiveness over these mandates, precipitating variation in adherence to stay-at-home orders [35]. Real-time metrics of mobility can provide better insight into population movement throughout the pandemic period than can be inferred from policy implementation alone.
Estimating the near-real-time effects of COVID-19 mobility restrictions on population mental health and suicide-related concerns is challenging given long delays in the availability of survey, administrative, and vital records data. Accordingly, use of alternate sources of surveillance information such as Google search volumes has increased during the initial phases of the pandemic. Several studies have examined Google search terms as an indicator of trends in mental health outcomes and suicidal behavior, both in the US and elsewhere [36–41]. Notably, attempts to validate search volumes for suicide-related terms using suicide deaths have found heterogeneous associations [40, 41], suggesting that search volumes also capture general inquires beyond acute suicidal crises that may reflect media coverage or anomalous search trends [42–44].
Nevertheless, since the beginning of the pandemic, many studies have assessed trends in economic, mental health, and suicide-related Google search volumes across countries [26, 45–55]. Overall, search volumes for specific terms related to mental health and suicide declined in the US and elsewhere during the period in which COVID-19 cases and restrictions on gathering increased. Relative search volumes for economic stressors possibly linked to suicide risk (e.g., unemployment), however, exponentially increased in the US during the early pandemic period of March through mid-April 2020 [26]. To our knowledge, studies of search volumes in the US have not incorporated data for longer periods despite additional waves of COVID-19. Further, exploring the impact of the pandemic on Google search volumes in the US has been hindered by methodological limitations, such as selecting a single time point (e.g., date of stay-at-home order) to define complex exposures [46–48] or resorting to descriptive analyses or correlations [26, 49–51]. To date, no study has implemented time-series analyses (TSA) to explore the relationship between a dynamic proxy of the personal impacts of the pandemic, such as mobility, and economic, mental health, or suicide-related search volumes [46, 48, 54, 55].
Trends in national mental health and economic concerns may also obscure the heterogeneity in pandemic experiences in specific areas. New York City (NYC) experienced especially high case and death rates early in the pandemic, with concomitant restrictions and stay-at-home orders that led to rising unemployment rates [56–58]. It is therefore important to understand the degree to which mobility restrictions influenced search volumes within such highly affected areas where stress and trauma are anticipated to be most pronounced.
The present study investigates the relationship between cell phone mobility data and Google Health Trends (GHT) search volumes related to social and economic stress, mental health, and suicide both nationally and in the NYC designated market area (DMA) from January 2020 through January 2021, capturing the emergence and evolution of the COVID-19 pandemic across the US [59]. We conducted TSA using transfer function models [40, 60, 61] to estimate associations between mobility and search volumes in two aims. First, given that stark increases in unemployment during the past year coincided with COVID-19 lockdowns across the nation, we examined the association between trends in mobility and trends in search volumes for economic stressors as a proof of principle to establish whether a signal could be detected. We anticipated that restricted mobility would portend increases in search volumes for economic stressors, which importantly may have lasting negative effects on mental health [19, 20, 31]. Second, we assessed the impact of mobility on search volumes for mental health and suicide-related terms. If, as predicted, search volumes for economic stressors were correlated with mobility, we postulated that search volumes for mental health or suicide seeking terms might then serve as valid indicators of mental health and suicidal behavior risk. We expected mild associations between mental health and suicide-related search volumes and mobility given prior heterogenous findings on the psychological effects of lockdowns [13].
Materials and methods
Economic stress, mental health, and suicide-related search volumes
We extracted weekly data from the GHT application programming interface from January 6, 2019 through January 23, 2021 for primary and extended sensitivity analyses (see below for details). GHT provides the likelihood for a specified term to be queried from a location, during a period [62, 63]. This is estimated as the number of searches for the term per 10 million searches from the specified period and location. We selected a weekly, rather than daily, time resolution in order to ensure that we had sufficient search volumes across term categories. In addition to documenting national trends in search volumes, we used search volumes from the NYC DMA. DMAs correspond to geographic areas that comprise the catchment area of local television viewing [64], and are the smallest geographic unit for which GHT data are available.
Search terms
We utilized the search term strategy outlined by Lee (2020) [41]. Terms covered a wide range of mental health and suicide-related categories including suicide neutral (e.g., “suicides”), suicide seeking (e.g., “how to kill myself”), suicide prevention (e.g., “suicide hotline”), mood and anxiety disorders (e.g., “depressed”), and psychosis (e.g., “delusion”). In order to ascertain the financial impact of the pandemic on search volumes, we divided Lee’s stressors and trauma term category into two mutually exclusive groups: economic stressors (e.g., “unemployed”) and social stressors (e.g., “lonely”, “posttraumatic stress disorder”), the latter of which we classified with mental health-related categories. Lee (2020) provides an overview of the association between these categories of search terms and suicide mortality in the US from January 2004 to December 2017 at 1 to 3-month time lags, finding heterogenous associations between search volumes and suicide mortality with further evidence of both positive and negative associations and sex differences. Given that some associations emerged for each category of searches, we used the Lee (2020) terms as they are comprehensive (S1 Table).
Mobility metrics
Daily data on mobility both nationally and by DMA were drawn from SafeGraph [65, 66]. These data include a panel of global positioning system pins from millions of mobile devices that are measured daily and available at the census-tract level. We aggregated these data by county that we then matched to the geographic area of the NYC DMA (see S2 Table). We used two measures from SafeGraph to estimate mobility: the median time (in minutes) mobile devices were at home on a given day and the proportion of devices in a given area that did not leave their home on a given day. SafeGraph estimates ‘home’ by the common nighttime location of a device within a 153x153 meter area. We restricted the end of our study period to exclude subsequent anomalous mobility datapoints.
Statistical analysis
We followed the methods described by Tran et al. (2017) on transfer function models [40, 61]. These methods permit assessment of the impact of our explanatory time series, mobility indicators, on our dependent time series, search volumes, while controlling for potential spurious correlations between the two. To do so, we first modeled our explanatory time series using autoregressive integrated moving average (ARIMA) models based on Box and Jenkins methods [60]. Daily mobility indicator data were averaged to match the weekly resolution of GHT data. We auto-selected the best fitting and most parsimonious models with the forecast package in R [67], which uses a variation of the Hyndman-Khandakar algorithm, returning the best model according to corrected Akaike information criterion. This method tests the benefit of including a seasonal component in the models using a Seasonal and Trend decomposition using Loess [67]. None of the optimal model forms selected by this method had a seasonal component. We then applied these models to the dependent time series and assessed if the two models’ residuals were correlated using the TSA package in R [61]. This process is termed prewhitening and any remaining correlations between models’ residuals indicate that changes in the dependent time series may be attributable to changes in the explanatory time series and vice versa.
For the primary analyses, we calculated correlations between the residuals of prewhitened explanatory and dependent time series at various time lags from January 5, 2020 through January 23, 2021 using the cross-correlation function. We visually inspected the residuals of the prewhitened time series to ensure that they met the assumptions of the normalized cross-correlation function, which approximates a time-dependent Pearson correlation coefficient [68], finding them to be independent, homoscedastic, and normally distributed [69]. Cross-correlations were assessed between time series at -14 to +14 week lags, including a 0-week lag. This range was auto-calculated based on the number of observations and time series in the data [61] and allowed for assessment of trends over several months, as recommended by Tran et al. (2017) [40]. Cross-correlation coefficients were categorized as small (.10 ≤ |r| < .30), medium (.30 ≤|r| < .50), or large (.50 ≥ |r|)) effect sizes [70]. P-values were calculated to quantify the strength of association against the null hypothesis; however they are susceptible to bias and should not be interpreted as a threshold for meaningful effects [71, 72]. A cross-correlation coefficient at a negative weekly lag (e.g. -10) indicates that changes in the explanatory time series lead changes in the dependent time series 10 weeks later. If a cross-correlation coefficient is at a positive time lag (e.g. 10), however, then it can be inferred that changes in the dependent time series lead changes in the explanatory time series 10 weeks later. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series. We anticipated, for example, to find larger positive cross-correlation coefficients at negative time lags between mobility indicators (explanatory) and economic stressor search volumes (dependent), meaning that decreased mobility predicted increased search volumes for terms related to economic stressors in future weeks. Cross-correlation coefficients at the 0-week lag suggest that changes in both time series were concurrent and could not be temporally disentangled.
Sensitivity analyses
We conducted TSA sensitivity analyses using data from January 6, 2019, the earliest available month of mobility data, through January 23, 2021, in order to assess whether findings were robust to the inclusion of pre-pandemic data. Because of the extended time period, cross-correlations were auto-assessed for a broader range of weekly time lags than in primary analyses, ranging from -17 to 17 weeks.
All statistical analyses were completed in in R version 4.0.3 [73]. Data were anonymized and exempt from human subjects approval.
Results
Mobility metrics
As illustrated in Fig 1, the proportion of devices completely at home rose sharply during March of 2020 to a daily peak in April of nearly 50% nationally and nearly 60% in the NYC DMA. During this period, the median daily amount of time that devices spent at home spiked to just over 18 hours (1,089 minutes, Interquartile Range (IQR): 859–1,245) nationally and over 21 hours (1,283 minutes, IQR: 936–1,409) in the NYC DMA. Both mobility measures declined over the course of the summer. Time at home reached its stable pre-pandemic levels, while the proportion of devices completely at home stabilized at slightly higher than its early-2020 levels. Results for the NYC DMA mirrored national trends, featuring more extreme peaks in the spring of 2020.
Note: Mobility data was aggregated to the weekly-level for time-series analyses.
Economic stress, mental health, and suicide-related search volumes
Fig 2 shows that search volumes (normalized per 10 million searches in a given area during a given week) for economic stressor terms rose both nationally and in the NYC DMA during March of 2020 from less than 5,000 searches per 10 million in early March to over 60,000 in late March nationally and nearing 80,000 in the NYC DMA. Search volumes for economic stressor terms then declined through the end of October, at which point they began to rise again through the beginning of 2021, coinciding with the fall and winter wave of new COVID-19 cases.
Fig 3 highlights results for suicide seeking, mood and anxiety, and social stressor term categories (S1 Fig includes results for all other search term categories). Trends remained relatively stable during the study period for terms in the suicide seeking, suicide neutral, suicide prevention, psychosis, and mood and anxiety categories both nationally and in the NYC DMA. Similarly, social stressor terms were generally consistent over time. Temporal fluctuations in search volumes for these term categories were minor compared to those of the economic stressor terms, although all term categories exhibited a dip during the emergence of the pandemic in the middle of March 2020.
Time series analyses (TSA)
We created heatmaps to depict TSA cross-correlation coefficients, for which numerical values and corresponding p-values can be found in S3 and S4 Tables. The algorithm selected best fit ARIMA models for each explanatory mobility time series in both primary and sensitivity analyses. Details on model parameters can be found in S5 Table. Values are reported below as (cross-correlation coefficient, p-value).
Economic TSA
Fig 4 presents the association between mobility and search volumes for economic stressor terms (e.g., “lost job”) that we examined as a proof of principle, given that we know that there was a strong connection between the onset of pandemic-related mobility limitations and economic stress. Larger positive cross-correlation coefficients for economic stressor terms were found at the -1 and 0-week lags for mobility indicators nationally (0.30, 0.027 to 0.60, <0.001), demonstrating that as mobility restrictions increased, search volumes for economic stressor terms increased both concurrently and one week later. Additionally, there were medium positive cross-correlation coefficients at weekly lags -11 (0.34, 0.011) and -6 (0.32, 0.018) and medium to large negative cross-correlation coefficients for weekly lags -9 (-0.35, 0.01) and -8 (-0.51, <0.001) for time at home nationally. In the NYC DMA, economic stressors were positively cross-correlated with time at home at a 1-week (0.29, 0.029) and with proportion of devices completely at home at -2 (0.27, 0.049) and 0-week (0.49, <0.001) lags.
P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.
The largest cross-correlation coefficient occurred with proportion of devices at home nationally at a 0-week lag (0.6, <0.001). This was also the largest cross-correlation coefficient across all analyses, indicating that severe mobility restriction and economic stressor search volumes during the pandemic period were strongly correlated but temporally inextricable. Findings of some larger negative cross-correlation coefficients complicated trend interpretation; however, overall, coefficients were predominantly positive and were strongest at the 0-week lag, suggesting that search volumes for economic stressors rose concurrently with pandemic mobility restrictions both nationally and in the NYC DMA.
Mental health and suicide TSA
Fig 5 presents heatmaps of primary analyses cross-correlation coefficients for suicide seeking, mood and anxiety, and social stressor search volumes. S2 Fig features suicide neutral, suicide prevention, and psychosis search volumes. Trends for these search term categories diverged by location, with stronger and more consistent national associations. Nationally, a series of medium to large positive cross-correlation coefficients at a 0-week lag for time at home emerged across term categories (0.37, 0.006 to 0.53, <0.001), excluding suicide neutral, and were buffered by smaller positive associations at adjacent time lags. These results broadly indicate that as mobility decreased (i.e. more time at home), search volumes for terms within these categories increased at the same time. Similarly, for proportion of devices completely at home nationally, smaller positive cross-correlation coefficients were found at the -9-week lag for suicide seeking, mood and anxiety, and suicide prevention search volumes (0.28, 0.041 to 0.35, 0.01); however, these findings were flanked by smaller negative cross-correlation coefficients which obfuscated trend interpretation. Despite differences across mobility indicators, these results suggest the declining mobility prompted increased searches for mental health and suicide-related terms during the pandemic.
P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.
Within the NYC DMA, however, there were few discernable patterns in cross-correlation coefficients for mental health and suicide-related terms across mobility indicators. Larger cross-correlation coefficients were irregularly distributed across negative and positive weekly lags and vacillated in direction. For time at home, only suicide seeking terms exhibited larger cross-correlation coefficients which alternated between positive and negative coefficients at weekly lags -3,-2, and 0 (-0.35, 0.009 to 0.42, 0.002). An analogous pattern also arose for suicide seeking search terms with proportion of devices completely at home at the -7 (-0.41, 0.002) and -6-week lags (0.3, 0.024). These alternating findings denote an inexplicable and potentially immaterial relationship between mobility and searches for suicide-seeking terms in the NYC DMA.
Sensitivity analyses
S3 Fig includes all heatmaps for sensitivity analyses with an extended study period of 2019 through 2021. As in primary analyses, we found larger, predominantly positive cross-correlation coefficients between economic stressor search volumes and mobility indicators across locations (0.21, 0.028 to 0.51, <0.001). These findings, however, spanned negative and positive weekly lags, signaling a bidirectional relationship between economic stress and mobility during this period.
Cross-correlation coefficients in sensitivity analyses were smaller and somewhat matched trends in primary analyses for mental health and suicide-related terms. Nationally, relatively larger positive cross-correlation coefficients for all search term categories except for suicide neutral and suicide seeking occurred at the -1-week lag between proportions of devices completely at home (0.19, 0.045 to 0.28, 0.004). As in primary analyses, however, these positive coefficients were immediately followed by smaller negative coefficients at the 0-week lag. Similarly, a string of negative cross-correlation coefficients for time at home at a 4-week lag were bound by positive correlations at adjacent lags. Ultimately, relationships, were mixed within term categories, especially between suicide seeking terms and time at home. Within the NYC DMA, findings were similarly ambiguous with no clear trends in cross-correlation coefficients.
Overall, the majority of analyses revealed positive concurrent associations between restricted mobility and search volumes for economic stressors. From 2020 through 2021 nationally, positive relationships were found between mobility restriction and mental health and suicide-related search volumes, excluding suicide neutral, and were strongest at the 0-week lag. Similar trends were not apparent within the NYC DMA.
Discussion
We employed mobility data documenting the proportion of cellular devices at home and time at home during the COVID-19 pandemic in the US and in New York City to assess whether restricted movement was associated with economic stress, mental health, and suicide as measured by Google Health Trends search volumes. Our research yielded two main findings. First, we found that pandemic mobility restrictions were contemporaneously linked to trajectories of online searches for terms related to economic distress across locations. Second, we found positive associations between declining mobility and mental health and suicide-related search volumes nationally during the pandemic. These results underscore the pervasiveness of pandemic-induced economic hardship and prompt further consideration of the relationships between isolation and mental health and suicide risk.
While Google data on mental health and suicide-related search volumes have been used as interim proxies for delayed epidemiological data during the pandemic [26, 45–55], to our knowledge, no studies have assessed a time-variant indicator of population behavior, such as mobility, as an exposure. Cellular mobility data provide real-time information on actual mobility restrictions [33, 34], a likely important driver of pandemic-related negative mental health outcomes [74, 75]. We found that mobility declines were most notable during the early spring of 2020 when NYC became a global center of the pandemic [76] and stay-at-home orders and social distancing recommendations were first established [77]. We also found that search volumes for economic stressor terms rapidly increased during the early stages of the pandemic, in accordance with other studies [26, 49], and in line with evidence documenting high rates of unemployment and financial hardship in response to pandemic-related economic shut-downs [78, 79]. Moreover, declining mobility was strongly correlated with concurrent and future rises in search volumes for economic stressor terms across locations, especially from 2020 through 2021, confirming our hypothesis that mobility restrictions due to travel limitations and quarantine orders would predict economic distress due to consequent economic adversity and job loss.
Having established a proof of principle, we found that search volumes for mental health and suicide-related terms declined slightly during the pandemic onset in early 2020, matching existing research [26, 46–51], and that mobility restrictions during January 2020 through January 2021 were associated with increased search volumes for these terms nationally, but not in the NYC DMA. Nationally, we identified strong concurrent relationships between mental health and suicide-related search volumes with time at home, which may be a more sensitive indicator of pandemic-induced isolation as it relates to declining psychiatric wellbeing. These findings align with longitudinal and cross-sectional studies in the US indicating that the prevalence of mental disorders and psychological distress increased during the early stages of the pandemic [1–8]. Findings of smaller correlations at heterogenous time lags that dissipated in geographically specific analyses and with the inclusion of pre-pandemic data are also consistent with evidence that lockdowns themselves have minor or mixed effects on population mental health [13, 80]. Further, provisional CDC estimates indicate that age-adjusted suicide rates during the first half of 2020 declined from those in 2019 [23]; however, emergency department visits related to suicide attempts for girls significantly increased over the course of the pandemic into 2021 [81], possibly explaining the inconsistent relationship between pandemic mobility restrictions and suicide seeking search volumes.
Foremost, our findings for economic stressor search volumes clarify the importance of financial concerns during the pandemic and highlight an opportunity to help those potentially vulnerable to medium- and long-term deteriorating mental health and increasing suicidal risk [31, 82]. Robust social safety net policies can mitigate the harms of prolonged financial distress [83, 84] and may contribute to ameliorating suicide risk [19]. Struggling individuals psychologically benefited from additional funds issued by the national government during the pandemic [85, 86] and, given the political will, governments can continue to grow and fortify the social safety net in the US [78].
The pandemic has presented many challenges to psychiatric epidemiology data collection, especially when administrative clinical service data and mortality records are lagged in their release. Our findings also contribute to the ongoing discussion surrounding the utility of Google search volumes in estimating mental health and suicide-related outcomes. Results for economic stressor, mental health, and, to some extent, suicide seeking terms matched trends in survey data. Yet attempts to assess the validity of Google search volumes, particularly in forecasting suicide deaths, [36–41] have concluded that search volumes for suicide-related terms, even when surveying granular term categories, are noisy and inconsistent indicators of suicide rates. Knipe et al. (2021, preprint) [45] analyzed the relationship between search volumes for mental health terms and mental health outcomes in the United Kingdom during the pandemic, also finding that the time series did not generally align. Ultimately, searches within the Google platform, with over 267 million unique users in the US [87], do not exactly measure population mental health outcomes and are not comparable with survey data, but instead may serve as a useful barometer for real-time population thoughts and concerns.
The extent to which Google search volumes are useful for mental health and suicide risk surveillance purposes remains unclear. For instance, search volumes for suicide-related terms often spike after a celebrity suicide death [37], which may indicate news story interest rather than increased psychological distress. Suicide deaths, however, also increase after a celebrity dies by suicide [88]. This is an example of the “Werther effect” which postulates that such increases in emulative or imitative suicide deaths among the public result from heightened news media coverage and personal identification with the celebrity [89]. Notably, Google search volumes cannot be disaggregated by social groups, such as race, although minority racial groups emerged as at growing risk of suicide mortality during the early phases of the pandemic [90, 91]. Yet despite their coarseness, spikes in search volumes for suicide-related terms may serve as early indicators of actual increases in local suicidal behavior risk, making continued surveillance warranted for public health efforts.
This study included several limitations; foremost that both cellular mobility and GHT search volumes are imperfect proxies for actual population behavior and mental health. Notably, other factors, including rates of COVID-19 illness, hospitalizations, and deaths or heightened attention to mass media, may be strongly related to mental health and suicide-related search volumes and should be explored in future analyses. Further, GHT search volumes could not be stratified by key at-risk demographic groups or adjusted for other potentially contemporaneous events that may have affected time series. We also may have been able to model and assess long-term trends and seasonality more rigorously if protracted pre-pandemic mobility data were available. Due to a lack of contemporaneous suicide mortality data, we were unable to assess the validity of suicide-related search volumes and our results may not be generalizable beyond the US.
In conclusion, we found that GHT search volumes captured real-time increases in economic distress that were temporally linked to COVID-19 pandemic mobility restrictions in the US and in New York City. Associations were also found between declining mobility and rising mental health and suicide-related search volumes nationally. Our results align with pandemic-period research exploring Google search volumes in the US and available data on unemployment, mental health, and suicide risk. As epidemiological studies strive to keep pace with the psychological impacts of the pandemic, we believe that Google search volumes may be a useful interim data source for monitoring population mental health when carefully considered.
Supporting information
S1 Fig. Google Health Trends search volumes for suicide neutral, suicide prevention, and psychosis terms over time nationally and in the NYC DMA: 2020–2021.
https://doi.org/10.1371/journal.pone.0260931.s001
(TIF)
S2 Fig. Heatmaps of cross-correlation coefficients for mobility indicators and Google Health Trends search volumes for suicide neutral, suicide prevention, and psychosis terms nationally and in the NYC DMA: 2020–2021.
P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.
https://doi.org/10.1371/journal.pone.0260931.s002
(TIF)
S3 Fig. Heatmaps of cross-correlation coefficients for mobility indicators and Google Health Trends search volumes for all search term categories nationally and in the NYC DMA: 2019–2021.
P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.
https://doi.org/10.1371/journal.pone.0260931.s003
(TIF)
S1 Table. Google search terms sourced from Lee (2020) [41].
https://doi.org/10.1371/journal.pone.0260931.s004
(PDF)
S2 Table. Counties matched to the New York City designated market area.
FIPS = Federal Information Processing Standards.
https://doi.org/10.1371/journal.pone.0260931.s005
(PDF)
S3 Table. Cross-correlation coefficients and p-values for mobility indicators Google Health Trends search volumes for all search term categories at weekly lags: 2020–2021.
P-value in parenthesis.
https://doi.org/10.1371/journal.pone.0260931.s006
(PDF)
S4 Table. Cross-correlation coefficients and p-values for mobility indicators and Google Health Trends search volumes for all search term categories at weekly lags: 2019–2021.
P-value in parenthesis.
https://doi.org/10.1371/journal.pone.0260931.s007
(PDF)
S5 Table. Fitted model parameters to mobility indicators nationally and in the NYC DMA.
ARIMA = autoregressive integrated moving average model. Non-seasonal ARIMA parameters = (p,d,q). p = autoregressive model order, d = degree of differencing, q = moving average model order.
https://doi.org/10.1371/journal.pone.0260931.s008
(PDF)
Acknowledgments
Mobility data was provided by Safe Graph, a data company that aggregates anonymized location data from numerous applications in order to provide insights about physical places, via the Placekey Community. To enhance privacy, SafeGraph excludes census block group information if fewer than five devices visited an establishment in a month from a given census block group.
References
- 1. Wu T, Jia X, Shi H, Niu J, Yin X, Xie J, et al. Prevalence of mental health problems during the COVID-19 pandemic: A systematic review and meta-analysis. Journal of Affective Disorders. 2021. pp. 91–98. pmid:33310451
- 2. Daly M, Sutin AR, Robinson E. Depression reported by US adults in 2017–2018 and March and April 2020. J Affect Disord. 2021;278: 131–135. pmid:32956962
- 3. Bueno-Notivol J, Gracia-García P, Olaya B, Lasheras I, López-Antón R, Santabárbara J. Prevalence of depression during the COVID-19 outbreak: A meta-analysis of community-based studies. Int J Clin Heal Psychol. 2021;21. pmid:32904715
- 4. Riehm KE, Holingue C, Smail EJ, Kapteyn A, Bennett D, Thrul J, et al. Trajectories of mental distress among U.S. adults during the COVID-19 pandemic. Ann Behav Med. 2021. pmid:33555336
- 5. Ettman CK, Abdalla SM, Cohen GH, Sampson L, Vivier PM, Galea S. Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic. JAMA Netw open. 2020;3: e2019686. pmid:32876685
- 6. Czeisler MÉ, Lane RI, Petrosky E, Wiley JF, Christensen A, Njai R, et al. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic—United States, June 24–30, 2020. MMWR Morb Mortal Wkly Rep. 2020;69: 1049–1057. pmid:32790653
- 7. Holingue C, Kalb LG, Riehm KE, Bennett D, Kapteyn A, Veldhuis CB, et al. Mental distress in the United States at the beginning of the COVID-19 pandemic. Am J Public Health. 2020;110: 1628–1634. pmid:32941066
- 8. Aknin L, Neve J-E De, Dunn E, Fancourt D, Goldberg E, Helliwell J, et al. Mental health during the first year of the COVID-19 pandemic: A review and recommendations for moving forward. PsyArXiv [Preprint]. 2021.
- 9. Shear MK, Ghesquiere A, Glickman K. Bereavement and complicated grief. Curr Psychiatry Rep. 2013;15: 406. pmid:24068457
- 10. Guldin MB, Ina Siegismund Kjaersgaard M, Fenger-Grøn M, Thorlund Parner E, Li J, Prior A, et al. Risk of suicide, deliberate self-harm and psychiatric illness after the loss of a close relative: A nationwide cohort study. World Psychiatry. 2017;16: 193–199. pmid:28498584
- 11. Hare-Duke L, Dening T, de Oliveira D, Milner K, Slade M. Conceptual framework for social connectedness in mental disorders: Systematic review and narrative synthesis. J Affect Disord. 2019;245: 188–199. pmid:30396057
- 12. Stickley A, Koyanagi A. Loneliness, common mental disorders and suicidal behavior: Findings from a general population survey. J Affect Disord. 2016;197: 81–87. pmid:26971125
- 13. Prati G, Mancini AD. The psychological impact of COVID-19 pandemic lockdowns: A review and meta-analysis of longitudinal studies and natural experiments. Psychol Med. 2021;51: 201–211. pmid:33436130
- 14. Hawton K, Lascelles K, Brand F, Casey D, Bale L, Ness J, et al. Self-harm and the COVID-19 pandemic: A study of factors contributing to self-harm during lockdown restrictions. J Psychiatr Res. 2021;137: 437–443. pmid:33774538
- 15. Holland KM, Jones C, Vivolo-Kantor AM, Idaikkadar N, Zwald M, Hoots B, et al. Trends in US emergency department visits for mental health, overdose, and violence outcomes before and during the COVID-19 pandemic. JAMA Psychiatry. 2021;78: 372–379. pmid:33533876
- 16. Harmon KJ, Fliss MD, Marshall SW, Peticolas K, Proescholdbell SK, Waller AE. The impact of the COVID-19 pandemic on the utilization of emergency department services for the treatment of injuries. Am J Emerg Med. 2021;47: 187–191. pmid:33892334
- 17. Walsh K, McLaughlin KA, Hamilton A, Keyes KM. Trauma exposure, incident psychiatric disorders, and disorder transitions in a longitudinal population representative sample. J Psychiatr Res. 2017;92: 212–218. pmid:28531836
- 18. MacIsaac MB, Bugeja LC, Jelinek GA. The association between exposure to interpersonal violence and suicide among women: a systematic review. Aust N Z J Public Health. 2017;41: 61–69. pmid:27774704
- 19. Milner A, Page A, LaMontagne AD. Long-term unemployment and suicide: A systematic review and meta-analysis. PLoS One. 2013;8: e51333. pmid:23341881
- 20. Nordt C, Warnke I, Seifritz E, Kawohl W. Modelling suicide and unemployment: A longitudinal analysis covering 63 countries, 2000–11. The Lancet Psychiatry. 2015;2: 239–245. pmid:26359902
- 21. Taquet M, Geddes JR, Husain M, Luciano S, Harrison PJ. 6-month neurological and psychiatric outcomes in 236 379 survivors of COVID-19: A retrospective cohort study using electronic health records. The Lancet Psychiatry. 2021;8: 416–427. pmid:33836148
- 22. Mazza MG, De Lorenzo R, Conte C, Poletti S, Vai B, Bollettini I, et al. Anxiety and depression in COVID-19 survivors: Role of inflammatory and clinical predictors. Brain Behav Immun. 2020;89: 594–600. pmid:32738287
- 23.
Ahmand FB, Cisewski JA. Quarterly provisional estimates for selected indicators of mortality, 2018-Quarter 3 2020. National Center for Health Statistics. National Vital Statistics System, Vital Statistics Rapid Release Program; 2021.
- 24. Pirkis J, John A, Shin S, Delpozo-Banos M, Arya V, Analuisa-Aguilar P, et al. Suicide trends in the early months of the COVID-19 pandemic: An interrupted time-series analysis of preliminary data from 21 countries. The Lancet Psychiatry. 2021;8: 579–588. pmid:33862016
- 25. John A, Okolie C, Eyles E, Webb RT, Schmidt L, McGuiness LA, et al. The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: A living systematic review. F1000Research. 2020;9: 1097. pmid:33604025
- 26. Halford EA, Lake AM, Gould MS. Google searches for suicide and suicide risk factors in the early stages of the COVID-19 pandemic. PLoS One. 2020;15: e0236777. pmid:32706835
- 27. Reger MA, Stanley IH, Joiner TE. Suicide mortality and coronavirus disease 2019—A perfect storm? JAMA Psychiatry. 2020;77: 1093–1094. pmid:32275300
- 28.
John A, Pirkis J, Gunnell D, Appleby L, Morrissey J. Trends in suicide during the COVID-19 pandemic. The BMJ. BMJ Publishing Group; 2020. https://doi.org/10.1136/bmj.m4352 pmid:33184048
- 29. Kawohl W, Nordt C. COVID-19, unemployment, and suicide. The Lancet Psychiatry. 2020;7: 389–390. pmid:32353269
- 30. Rogers JP, Chesney E, Oliver D, Begum N, Saini A, Wang S, et al. Suicide, self-harm and thoughts of suicide or self-harm in infectious disease epidemics: A systematic review and meta-analysis. Epidemiol Psychiatr Sci. 2021;30: e32. pmid:33902775
- 31. Karanikolos M, Heino P, McKee M, Stuckler D, Legido-Quigley H. Effects of the global financial crisis on health in high-income OECD countries: A narrative review. Int J Heal Serv. 2016;46: 208–240. pmid:27076651
- 32. Tanaka T, Okamoto S. Increase in suicide following an initial decline during the COVID-19 pandemic in Japan. Nat Hum Behav. 2021;5: 229–238. pmid:33452498
- 33. Lee M, Zhao J, Sun Q, Pan Y, Zhou W, Xiong C, et al. Human mobility trends during the early stage of the COVID-19 pandemic in the United States. PLoS One. 2020;15: e0241468. pmid:33166301
- 34. Xiong C, Hu S, Yang M, Younes H, Luo W, Ghader S, et al. Mobile device location data reveal human mobility response to state-level stay-at-home orders during the COVID-19 pandemic in the USA. J R Soc Interface. 2020;17: 20200344. pmid:33323055
- 35. Allcott H, Boxell L, Conway J, Gentzkow M, Thaler M, Yang D. Polarization and public health: Partisan differences in social distancing during the coronavirus pandemic. J Public Econ. 2020;191: 104254. pmid:32836504
- 36. Barros JM, Melia R, Francis K, Bogue J, O’Sullivan M, Young K, et al. The validity of Google Trends search volumes for behavioral forecasting of national suicide rates in Ireland. Int J Environ Res Public Health. 2019;16: 3201. pmid:31480718
- 37. Gunn Iii JF, Goldstein SE, Lester D. The impact of widely publicized suicides on search trends: Using Google Trends to test the Werther and Papageno effects. Arch Suicide Res. 2020;24: 142–155. pmid:30300114
- 38. Solano P, Ustulin M, Pizzorno E, Vichi M, Pompili M, Serafini G, et al. A Google-based approach for monitoring suicide risk. Psychiatry Res. 2016;246: 581–586. pmid:27837725
- 39.
Akers RL, Jensen GF. The empirical status of social learning theory of crime and deviance: The past, present, and future. Taking stock: The status of criminological theory. Piscataway, NJ, US: Transaction Publishers; 2006. pp. 37–76.
- 40. Tran US, Andel R, Niederkrotenthaler T, Till B, Ajdacic-Gross V, Voracek M. Low validity of Google Trends for behavioral forecasting of national suicide rates. PLoS One. 2017;12: e0183149. pmid:28813490
- 41. Lee J-Y. Search trends preceding increases in suicide: A cross-correlation study of monthly Google search volume and suicide rate using transfer function models. J Affect Disord. 2020;262: 155–164. pmid:31733460
- 42. Cervellin G, Comelli I, Lippi G. Is Google Trends a reliable tool for digital epidemiology? Insights from different clinical settings. J Epidemiol Glob Health. 2017;7: 185. pmid:28756828
- 43. Rovetta A. Reliability of Google Trends: Analysis of the limits and potential of web infoveillance during COVID-19 pandemic and for future research. Front Res Metrics Anal. 2021;6. pmid:34113751
- 44. Dagher SH, Lamé G, Hubiche T, Ezzedine K, Duong TA. The influence of media coverage and governmental policies on Google queries related to COVID-19 cutaneous symptoms: Infodemiology study. JMIR Public Heal Surveill. 2021;7. pmid:33513563
- 45. Knipe D, Gunnell D, Evans H, John A, Fancourt D. Is Google Trends a useful tool for tracking mental and social distress during a public health emergency? A time-series analysis. medRxiv [Preprint]. 2021; 2021.02.18.21251966. pmid:34348169
- 46. Brodeur A, Clark AE, Fleche S, Powdthavee N. COVID-19, lockdowns and well-being: Evidence from Google Trends. J Public Econ. 2021;193: 104346. pmid:33281237
- 47. Jacobson NC, Lekkas D, Price G, Heinz M V, Song M, O’Malley AJ, et al. Flattening the mental health curve: COVID-19 stay-at-home orders Are associated with alterations in mental health search behavior in the United States. JMIR Ment Jealth. 2020;7: e19347–e19347. pmid:32459186
- 48. Sinyor M, Spittal MJ, Niederkrotenthaler T. Changes in suicide and resilience-related Google searches during the early stages of the COVID-19 pandemic. Canadian Journal of Psychiatry. 2020. pp. 741–743. pmid:32524848
- 49. Knipe D, Evans H, Marchant A, Gunnell D, John A. Mapping population mental health concerns related to COVID-19 and the consequences of physical distancing: a Google trends analysis. Wellcome Open Res. 2020;5: 82. pmid:32671230
- 50. Misiak B, Szcześniak D, Koczanowicz L, Rymaszewska J. The COVID-19 outbreak and Google searches: Is it really the time to worry about global mental health? Brain Behav Immun. 2020;87: 126–127. pmid:32360605
- 51. Hoerger M, Alonzi S, Perry LM, Voss HM, Easwar S, Gerhart JI. Impact of the COVID-19 pandemic on mental health: Real-time surveillance using Google Trends. Psychol Trauma Theory, Res Pract Policy. 2020;12: 567–568. pmid:32790441
- 52. Chen H, Zhang K. Insight into the psychological problems on the epidemic of COVID-19 in China by online searching behaviors. J Affect Disord. 2020;276: 1093–1094. pmid:32771861
- 53. Tennant P, Gonzalez J, Rowan M, Hilbelink C, Moore Q. The association between new COVID-19 cases and Google searches for mental health. Adv [Preprint]. 2020. https://doi.org/10.31124/advance.12375020.v2
- 54. Ayers JW, Leas EC, Johnson DC, Poliak A, Althouse BM, Dredze M, et al. Internet searches for acute anxiety during the early stages of the COVID-19 pandemic. JAMA Intern Med. 2020. pmid:32832984
- 55. Lin Y-H, Chiang T-W, Lin Y-L. Increased internet searches for insomnia as an indicator of global mental health during the COVID-19 pandemic: Multinational longitudinal study. J Med Internet Res. 2020;22: e22181. pmid:32924951
- 56.
Nischan U. Uneven impact: What job loss during COVID-19 means for New Yorkers now and into the future. Dec 2020 [cited 23 Mar 2021]. Available: https://www1.nyc.gov/assets/dca/downloads/pdf/partners/Uneven_Impact.pdf
- 57. Wadhera RK, Wadhera P, Gaba P, Figueroa JF, Joynt Maddox KE, Yeh RW, et al. Variation in COVID-19 hospitalizations and deaths across New York City boroughs. JAMA. American Medical Association; 2020. pp. 2192–2195. pmid:32347898
- 58. Borjas GJ. Business closures, stay-at-home restrictions, and COVID-19 testing outcomes in New York City. Prev Chronic Dis. 2020;17. pmid:32945766
- 59.
Taylor DB. A timeline of the Coronavirus pandemic. In: The New York Times [Internet]. 17 Mar 2021 [cited 16 Jun 2021]. Available: https://www.nytimes.com/article/coronavirus-timeline.html
- 60.
Box G, Jenkins G. Time series analysis: Forecasting and control. San Francisco: Holden-Day; 1970.
- 61.
Cryer JD, Chan K-S. Time series analysis: With applications in R. New York: Springer; 2008. https://doi.org/10.1007/978-0-387-75959-3
- 62. Stocking G, Matsa KE. Using Google Trends data for research? Here are 6 questions to ask. In: Medium [Internet]. 27 Apr 2017 [cited 11 Jun 2021]. Available: https://medium.com/@pewresearch/using-google-trends-data-for-research-here-are-6-questions-to-ask-a7097f5fb526
- 63. Nuti S V., Wayda B, Ranasinghe I, Wang S, Dreyer RP, Chen SI, et al. The use of Google Trends in health care research: A systematic review. PLoS One. 2014;9. pmid:25337815
- 64.
DMA Regions. In: Nielson [Internet]. 2021 [cited 24 Mar 2021]. Available: https://www.nielsen.com/us/en/intl-campaigns/dma-maps/
- 65.
SafeGraph. Social Distancing Metrics. 2021. Available: https://docs.safegraph.com/docs/social-distancing-metrics
- 66. Lamb MR, Kandula S, Shaman J. Differential COVID-19 case positivity in New York City neighborhoods: Socioeconomic factors and mobility. Influenza Other Respi Viruses. 2021;15: 209–217. pmid:33280263
- 67. Hyndman RJ, Khandakar Y. Automatic time series forecasting: The forecast package for R. J Stat Softw. 2008;27: 1–22.
- 68. Derrick T, Thomas J. Time series analysis: The cross-correlation function. Innovative Analyses of Human Movement. 2004. pp. 189–205.
- 69. Schober P, Schwarte LA. Correlation coefficients: Appropriate use and interpretation. Anesth Analg. 2018;126: 1763–1768. pmid:29481436
- 70. Cohen J. A power primer. Psychol Bull. 1992;112: 155–159. pmid:19565683
- 71. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, et al. Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. Eur J Epidemiol. 2016;31: 337. pmid:27209009
- 72. Amrhein V, Korner-Nievergelt F, Roth T. The earth is flat (p > 0.05): Significance thresholds and the crisis of unreplicable research. PeerJ. 2017;5. pmid:28698825
- 73.
R Core Team. R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing; 2021.
- 74. Pantelaki E, Maggi E, Crotti D. Mobility impact and well-being in later life: A multidisciplinary systematic review. Res Transp Econ. 2020;86: 100975.
- 75. Polku H, Mikkola TM, Portegijs E, Rantakokko M, Kokko K, Kauppinen M, et al. Life-space mobility and dimensions of depressive symptoms among community-dwelling older adults. Aging Ment Heal. 2015;19: 781–789. pmid:25376479
- 76.
McKinley J. New York City region is now an epicenter of the Coronavirus pandemic. In: The New York Times [Internet]. 22 Mar 2020 [cited 11 May 2021]. Available: https://www.nytimes.com/2020/03/22/nyregion/Coronavirus-new-York-epicenter.html
- 77.
Mervosh S, Lu D, Swales V. See which states and cities have told residents to stay at home. In: The New York Times [Internet]. 20 Apr 2020 [cited 11 May 2021]. Available: https://www.nytimes.com/interactive/2020/us/coronavirus-stay-at-home-order.html
- 78. Moffitt RA, Ziliak JP. COVID-19 and the US safety net. Fisc Stud. 2020;41: 515–548. pmid:33362310
- 79.
Parker K, Minkin R, Bennett J. Economic fallout from COVID-19 continues to hit lower-income Americans the hardest. In: Pew Research Center [Internet]. 24 Sep 2020 [cited 11 May 2021]. Available: https://www.pewresearch.org/social-trends/2020/09/24/economic-fallout-from-covid-19-continues-to-hit-lower-income-americans-the-hardest/
- 80. Lau JTF, Yang X, Tsui HY, Pang E, Wing YK. Positive mental health-related impacts of the SARS epidemic on the general public in Hong Kong and their associations with other negative impacts. J Infect. 2006;53: 114–124. pmid:16343636
- 81. Yard E, Radhakrishnan L, Ballesteros MF, Sheppard M, Gates A, Stein Z, et al. Emergency department visits for suspected suicide attempts among persons aged 12–25 years before and during the COVID-19 pandemic—United States, January 2019–May 2021. MMWR Morb Mortal Wkly Rep. 2021;70. pmid:34138833
- 82. Knapp EA, Bilal U, Dean LT, Lazo M, Celentano DD. Economic insecurity and deaths of despair in US counties. Am J Epidemiol. 2019;188: 2131–2139. pmid:31172197
- 83. Cylus J, Glymour MM, Avendano M. Do generous unemployment benefit programs reduce suicide rates? A state fixed-effect analysis covering 1968–2008. Am J Epidemiol. 2014;180: 45–52. pmid:24939978
- 84. Cylus J, Glymour MM, Avendano M. Health effects of unemployment benefit program generosity. Am J Public Health. 2015;105: 317–323. pmid:25521897
- 85. Berkowitz SA, Basu S. Unmet social needs and worse mental health after expiration of COVID-19 Federal Pandemic Unemployment Compensation. Health Aff (Millwood). 2021;40: 426–434. pmid:33600235
- 86. Berkowitz SA, Basu S. Unemployment insurance, health-related social needs, health care access, and mental health during the COVID-19 pandemic. JAMA Intern Med. 2020;181: 699–702. pmid:33252615
- 87.
Johnson J. Google—Statistics & Facts. In: Statista [Internet]. 25 Feb 2021 [cited 11 May 2021]. Available: https://www.statista.com/topics/1001/google/#dossierSummary
- 88. Fink DS, Santaella-Tenorio J, Keyes KM. Increase in suicides the months after the death of Robin Williams in the US. PLoS One. 2018;13. pmid:29415016
- 89. Phillips DP. The influence of suggestion on suicide: Substantive and theroretical implications of the Werther effect. Am Sociol Rev. 1974;39: 340–354. pmid:4844994
- 90. Mitchell TO, Li L. State-level data on suicide mortality during COVID-19 quarantine: Early evidence of a disproportionate impact on racial minorities. Psychiatry Res. 2021;295: 113629. pmid:33290944
- 91. Bray MJC, Daneshvari NO, Radhakrishnan I, Cubbage J, Eagle M, Southall P, et al. Racial differences in statewide suicide mortality trends in Maryland during the Coronavirus disease 2019 (COVID-19) pandemic. JAMA Psychiatry. 2020. pmid:33325985