Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Associations between COVID-19 mobility restrictions and economic, mental health, and suicide-related concerns in the US using cellular phone GPS and Google search volume data

  • Catherine Gimbrone ,

    Roles Formal analysis, Methodology, Validation, Visualization, Writing – original draft

    c.gimbrone@columbia.edu

    Affiliation Department of Epidemiology, Columbia University, New York, NY, United States of America

  • Caroline Rutherford,

    Roles Formal analysis, Methodology, Validation, Visualization, Writing – review & editing

    Affiliation Department of Epidemiology, Columbia University, New York, NY, United States of America

  • Sasikiran Kandula,

    Roles Data curation, Software, Writing – review & editing

    Affiliation Department of Environmental Health Sciences, Columbia University, New York, NY, United States of America

  • Gonzalo Martínez-Alés,

    Roles Supervision, Writing – review & editing

    Affiliation Department of Epidemiology, Columbia University, New York, NY, United States of America

  • Jeffrey Shaman,

    Roles Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliation Department of Environmental Health Sciences, Columbia University, New York, NY, United States of America

  • Mark Olfson,

    Roles Supervision, Writing – review & editing

    Affiliations Department of Epidemiology, Columbia University, New York, NY, United States of America, Department of Psychiatry, Columbia University, New York, NY, United States of America

  • Madelyn S. Gould,

    Roles Supervision, Writing – review & editing

    Affiliations Department of Epidemiology, Columbia University, New York, NY, United States of America, Department of Psychiatry, Columbia University, New York, NY, United States of America

  • Sen Pei,

    Roles Writing – review & editing

    Affiliation Department of Environmental Health Sciences, Columbia University, New York, NY, United States of America

  • Marta Galanti,

    Roles Writing – review & editing

    Affiliation Department of Environmental Health Sciences, Columbia University, New York, NY, United States of America

  • Katherine M. Keyes

    Roles Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliation Department of Epidemiology, Columbia University, New York, NY, United States of America

Abstract

During the COVID-19 pandemic, US populations have experienced elevated rates of financial and psychological distress that could lead to increases in suicide rates. Rapid ongoing mental health monitoring is critical for early intervention, especially in regions most affected by the pandemic, yet traditional surveillance data are available only after long lags. Novel information on real-time population isolation and concerns stemming from the pandemic’s social and economic impacts, via cellular mobility tracking and online search data, are potentially important interim surveillance resources. Using these measures, we employed transfer function model time-series analyses to estimate associations between daily mobility indicators (proportion of cellular devices completely at home and time spent at home) and Google Health Trends search volumes for terms pertaining to economic stress, mental health, and suicide during 2020 and 2021 both nationally and in New York City. During the first pandemic wave in early-spring 2020, over 50% of devices remained completely at home and searches for economic stressors exceeded 60,000 per 10 million. We found large concurrent associations across analyses between declining mobility and increasing searches for economic stressor terms (national proportion of devices at home: cross-correlation coefficient (CC) = 0.6 (p-value <0.001)). Nationally, we also found strong associations between declining mobility and increasing mental health and suicide-related searches (time at home: mood/anxiety CC = 0.53 (<0.001), social stressor CC = 0.51 (<0.001), suicide seeking CC = 0.37 (0.006)). Our findings suggest that pandemic-related isolation coincided with acute economic distress and may be a risk factor for poor mental health and suicidal behavior. These emergent relationships warrant ongoing attention and causal assessment given the potential for long-term psychological impact and suicide death. As US populations continue to face stress, Google search data can be used to identify possible warning signs from real-time changes in distributions of population thought patterns.

Introduction

The SARS-CoV-2 pandemic and resulting illness and death related to COVID-19 are associated with deleterious effects on population mental health in the US [18]. There are several potential mechanisms through which the pandemic may affect mental health and suicide risk, including loss and bereavement of loved ones [9, 10], social isolation and loneliness due to quarantine and public health recommendations limiting mobility [1113], diminished access to mental healthcare and treatment within an increasingly overwhelmed medical system [1416], reduced ability of existing safety nets to mitigate abuse [17, 18], and unemployment and financial instability due to restrictions on business activity [19, 20]. Available data indicate increased rates of psychiatric disorders, such as depression [2, 7, 8], and exacerbated existing mental health disorders among COVID-19 patients in the first waves of the pandemic [21, 22]. Contrary to many hypotheses, however, provisional death data suggest that suicide mortality in the US did not increase during the initial pandemic outbreak [23, 24], yet there is widespread concern that rising financial insecurity and deteriorating mental health could lead to subsequent increases [2529]. Increased suicide rates have been observed after previous pandemics [30] and recessions [31], and in other areas of the world during 2020 [32]. Using contemporary information to monitor the dynamic medium- and long-term impacts of the pandemic on economic, mental health, and suicide-related concerns is critical to guiding prevention efforts.

Accurate assessment of the evolving effects of the pandemic on economic stress, mental health, and suicide risk requires a fluid measure of population behavior over time, such as mobility (e.g. leaving one’s home or neighborhood). While cities and states passed stay-at-home orders on specific dates that can be tracked, the psychological impacts of these policies remain unclear [13] and mobility declines often preceded their passage [33, 34]. The politicization of the pandemic also spurred divisiveness over these mandates, precipitating variation in adherence to stay-at-home orders [35]. Real-time metrics of mobility can provide better insight into population movement throughout the pandemic period than can be inferred from policy implementation alone.

Estimating the near-real-time effects of COVID-19 mobility restrictions on population mental health and suicide-related concerns is challenging given long delays in the availability of survey, administrative, and vital records data. Accordingly, use of alternate sources of surveillance information such as Google search volumes has increased during the initial phases of the pandemic. Several studies have examined Google search terms as an indicator of trends in mental health outcomes and suicidal behavior, both in the US and elsewhere [3641]. Notably, attempts to validate search volumes for suicide-related terms using suicide deaths have found heterogeneous associations [40, 41], suggesting that search volumes also capture general inquires beyond acute suicidal crises that may reflect media coverage or anomalous search trends [4244].

Nevertheless, since the beginning of the pandemic, many studies have assessed trends in economic, mental health, and suicide-related Google search volumes across countries [26, 4555]. Overall, search volumes for specific terms related to mental health and suicide declined in the US and elsewhere during the period in which COVID-19 cases and restrictions on gathering increased. Relative search volumes for economic stressors possibly linked to suicide risk (e.g., unemployment), however, exponentially increased in the US during the early pandemic period of March through mid-April 2020 [26]. To our knowledge, studies of search volumes in the US have not incorporated data for longer periods despite additional waves of COVID-19. Further, exploring the impact of the pandemic on Google search volumes in the US has been hindered by methodological limitations, such as selecting a single time point (e.g., date of stay-at-home order) to define complex exposures [4648] or resorting to descriptive analyses or correlations [26, 4951]. To date, no study has implemented time-series analyses (TSA) to explore the relationship between a dynamic proxy of the personal impacts of the pandemic, such as mobility, and economic, mental health, or suicide-related search volumes [46, 48, 54, 55].

Trends in national mental health and economic concerns may also obscure the heterogeneity in pandemic experiences in specific areas. New York City (NYC) experienced especially high case and death rates early in the pandemic, with concomitant restrictions and stay-at-home orders that led to rising unemployment rates [5658]. It is therefore important to understand the degree to which mobility restrictions influenced search volumes within such highly affected areas where stress and trauma are anticipated to be most pronounced.

The present study investigates the relationship between cell phone mobility data and Google Health Trends (GHT) search volumes related to social and economic stress, mental health, and suicide both nationally and in the NYC designated market area (DMA) from January 2020 through January 2021, capturing the emergence and evolution of the COVID-19 pandemic across the US [59]. We conducted TSA using transfer function models [40, 60, 61] to estimate associations between mobility and search volumes in two aims. First, given that stark increases in unemployment during the past year coincided with COVID-19 lockdowns across the nation, we examined the association between trends in mobility and trends in search volumes for economic stressors as a proof of principle to establish whether a signal could be detected. We anticipated that restricted mobility would portend increases in search volumes for economic stressors, which importantly may have lasting negative effects on mental health [19, 20, 31]. Second, we assessed the impact of mobility on search volumes for mental health and suicide-related terms. If, as predicted, search volumes for economic stressors were correlated with mobility, we postulated that search volumes for mental health or suicide seeking terms might then serve as valid indicators of mental health and suicidal behavior risk. We expected mild associations between mental health and suicide-related search volumes and mobility given prior heterogenous findings on the psychological effects of lockdowns [13].

Materials and methods

Economic stress, mental health, and suicide-related search volumes

We extracted weekly data from the GHT application programming interface from January 6, 2019 through January 23, 2021 for primary and extended sensitivity analyses (see below for details). GHT provides the likelihood for a specified term to be queried from a location, during a period [62, 63]. This is estimated as the number of searches for the term per 10 million searches from the specified period and location. We selected a weekly, rather than daily, time resolution in order to ensure that we had sufficient search volumes across term categories. In addition to documenting national trends in search volumes, we used search volumes from the NYC DMA. DMAs correspond to geographic areas that comprise the catchment area of local television viewing [64], and are the smallest geographic unit for which GHT data are available.

Search terms

We utilized the search term strategy outlined by Lee (2020) [41]. Terms covered a wide range of mental health and suicide-related categories including suicide neutral (e.g., “suicides”), suicide seeking (e.g., “how to kill myself”), suicide prevention (e.g., “suicide hotline”), mood and anxiety disorders (e.g., “depressed”), and psychosis (e.g., “delusion”). In order to ascertain the financial impact of the pandemic on search volumes, we divided Lee’s stressors and trauma term category into two mutually exclusive groups: economic stressors (e.g., “unemployed”) and social stressors (e.g., “lonely”, “posttraumatic stress disorder”), the latter of which we classified with mental health-related categories. Lee (2020) provides an overview of the association between these categories of search terms and suicide mortality in the US from January 2004 to December 2017 at 1 to 3-month time lags, finding heterogenous associations between search volumes and suicide mortality with further evidence of both positive and negative associations and sex differences. Given that some associations emerged for each category of searches, we used the Lee (2020) terms as they are comprehensive (S1 Table).

Mobility metrics

Daily data on mobility both nationally and by DMA were drawn from SafeGraph [65, 66]. These data include a panel of global positioning system pins from millions of mobile devices that are measured daily and available at the census-tract level. We aggregated these data by county that we then matched to the geographic area of the NYC DMA (see S2 Table). We used two measures from SafeGraph to estimate mobility: the median time (in minutes) mobile devices were at home on a given day and the proportion of devices in a given area that did not leave their home on a given day. SafeGraph estimates ‘home’ by the common nighttime location of a device within a 153x153 meter area. We restricted the end of our study period to exclude subsequent anomalous mobility datapoints.

Statistical analysis

We followed the methods described by Tran et al. (2017) on transfer function models [40, 61]. These methods permit assessment of the impact of our explanatory time series, mobility indicators, on our dependent time series, search volumes, while controlling for potential spurious correlations between the two. To do so, we first modeled our explanatory time series using autoregressive integrated moving average (ARIMA) models based on Box and Jenkins methods [60]. Daily mobility indicator data were averaged to match the weekly resolution of GHT data. We auto-selected the best fitting and most parsimonious models with the forecast package in R [67], which uses a variation of the Hyndman-Khandakar algorithm, returning the best model according to corrected Akaike information criterion. This method tests the benefit of including a seasonal component in the models using a Seasonal and Trend decomposition using Loess [67]. None of the optimal model forms selected by this method had a seasonal component. We then applied these models to the dependent time series and assessed if the two models’ residuals were correlated using the TSA package in R [61]. This process is termed prewhitening and any remaining correlations between models’ residuals indicate that changes in the dependent time series may be attributable to changes in the explanatory time series and vice versa.

For the primary analyses, we calculated correlations between the residuals of prewhitened explanatory and dependent time series at various time lags from January 5, 2020 through January 23, 2021 using the cross-correlation function. We visually inspected the residuals of the prewhitened time series to ensure that they met the assumptions of the normalized cross-correlation function, which approximates a time-dependent Pearson correlation coefficient [68], finding them to be independent, homoscedastic, and normally distributed [69]. Cross-correlations were assessed between time series at -14 to +14 week lags, including a 0-week lag. This range was auto-calculated based on the number of observations and time series in the data [61] and allowed for assessment of trends over several months, as recommended by Tran et al. (2017) [40]. Cross-correlation coefficients were categorized as small (.10 ≤ |r| < .30), medium (.30 ≤|r| < .50), or large (.50 ≥ |r|)) effect sizes [70]. P-values were calculated to quantify the strength of association against the null hypothesis; however they are susceptible to bias and should not be interpreted as a threshold for meaningful effects [71, 72]. A cross-correlation coefficient at a negative weekly lag (e.g. -10) indicates that changes in the explanatory time series lead changes in the dependent time series 10 weeks later. If a cross-correlation coefficient is at a positive time lag (e.g. 10), however, then it can be inferred that changes in the dependent time series lead changes in the explanatory time series 10 weeks later. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series. We anticipated, for example, to find larger positive cross-correlation coefficients at negative time lags between mobility indicators (explanatory) and economic stressor search volumes (dependent), meaning that decreased mobility predicted increased search volumes for terms related to economic stressors in future weeks. Cross-correlation coefficients at the 0-week lag suggest that changes in both time series were concurrent and could not be temporally disentangled.

Sensitivity analyses

We conducted TSA sensitivity analyses using data from January 6, 2019, the earliest available month of mobility data, through January 23, 2021, in order to assess whether findings were robust to the inclusion of pre-pandemic data. Because of the extended time period, cross-correlations were auto-assessed for a broader range of weekly time lags than in primary analyses, ranging from -17 to 17 weeks.

All statistical analyses were completed in in R version 4.0.3 [73]. Data were anonymized and exempt from human subjects approval.

Results

Mobility metrics

As illustrated in Fig 1, the proportion of devices completely at home rose sharply during March of 2020 to a daily peak in April of nearly 50% nationally and nearly 60% in the NYC DMA. During this period, the median daily amount of time that devices spent at home spiked to just over 18 hours (1,089 minutes, Interquartile Range (IQR): 859–1,245) nationally and over 21 hours (1,283 minutes, IQR: 936–1,409) in the NYC DMA. Both mobility measures declined over the course of the summer. Time at home reached its stable pre-pandemic levels, while the proportion of devices completely at home stabilized at slightly higher than its early-2020 levels. Results for the NYC DMA mirrored national trends, featuring more extreme peaks in the spring of 2020.

thumbnail
Fig 1. Mobility indicators over time nationally and in the NYC DMA: 2020–2021.

Note: Mobility data was aggregated to the weekly-level for time-series analyses.

https://doi.org/10.1371/journal.pone.0260931.g001

Economic stress, mental health, and suicide-related search volumes

Fig 2 shows that search volumes (normalized per 10 million searches in a given area during a given week) for economic stressor terms rose both nationally and in the NYC DMA during March of 2020 from less than 5,000 searches per 10 million in early March to over 60,000 in late March nationally and nearing 80,000 in the NYC DMA. Search volumes for economic stressor terms then declined through the end of October, at which point they began to rise again through the beginning of 2021, coinciding with the fall and winter wave of new COVID-19 cases.

thumbnail
Fig 2. Google Health Trends search volumes for economic stressor terms over time nationally and in the NYC DMA: 2020–2021.

https://doi.org/10.1371/journal.pone.0260931.g002

Fig 3 highlights results for suicide seeking, mood and anxiety, and social stressor term categories (S1 Fig includes results for all other search term categories). Trends remained relatively stable during the study period for terms in the suicide seeking, suicide neutral, suicide prevention, psychosis, and mood and anxiety categories both nationally and in the NYC DMA. Similarly, social stressor terms were generally consistent over time. Temporal fluctuations in search volumes for these term categories were minor compared to those of the economic stressor terms, although all term categories exhibited a dip during the emergence of the pandemic in the middle of March 2020.

thumbnail
Fig 3. Google Health Trends search volumes for suicide seeking, mood and anxiety, and social stressor terms over time nationally and in the NYC DMA: 2020–2021.

https://doi.org/10.1371/journal.pone.0260931.g003

Time series analyses (TSA)

We created heatmaps to depict TSA cross-correlation coefficients, for which numerical values and corresponding p-values can be found in S3 and S4 Tables. The algorithm selected best fit ARIMA models for each explanatory mobility time series in both primary and sensitivity analyses. Details on model parameters can be found in S5 Table. Values are reported below as (cross-correlation coefficient, p-value).

Economic TSA

Fig 4 presents the association between mobility and search volumes for economic stressor terms (e.g., “lost job”) that we examined as a proof of principle, given that we know that there was a strong connection between the onset of pandemic-related mobility limitations and economic stress. Larger positive cross-correlation coefficients for economic stressor terms were found at the -1 and 0-week lags for mobility indicators nationally (0.30, 0.027 to 0.60, <0.001), demonstrating that as mobility restrictions increased, search volumes for economic stressor terms increased both concurrently and one week later. Additionally, there were medium positive cross-correlation coefficients at weekly lags -11 (0.34, 0.011) and -6 (0.32, 0.018) and medium to large negative cross-correlation coefficients for weekly lags -9 (-0.35, 0.01) and -8 (-0.51, <0.001) for time at home nationally. In the NYC DMA, economic stressors were positively cross-correlated with time at home at a 1-week (0.29, 0.029) and with proportion of devices completely at home at -2 (0.27, 0.049) and 0-week (0.49, <0.001) lags.

thumbnail
Fig 4. Heatmaps of cross-correlation coefficients for mobility indicators and Google Health Trends search volumes for economic stressor terms nationally and in the NYC DMA: 2020–2021.

P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.

https://doi.org/10.1371/journal.pone.0260931.g004

The largest cross-correlation coefficient occurred with proportion of devices at home nationally at a 0-week lag (0.6, <0.001). This was also the largest cross-correlation coefficient across all analyses, indicating that severe mobility restriction and economic stressor search volumes during the pandemic period were strongly correlated but temporally inextricable. Findings of some larger negative cross-correlation coefficients complicated trend interpretation; however, overall, coefficients were predominantly positive and were strongest at the 0-week lag, suggesting that search volumes for economic stressors rose concurrently with pandemic mobility restrictions both nationally and in the NYC DMA.

Mental health and suicide TSA

Fig 5 presents heatmaps of primary analyses cross-correlation coefficients for suicide seeking, mood and anxiety, and social stressor search volumes. S2 Fig features suicide neutral, suicide prevention, and psychosis search volumes. Trends for these search term categories diverged by location, with stronger and more consistent national associations. Nationally, a series of medium to large positive cross-correlation coefficients at a 0-week lag for time at home emerged across term categories (0.37, 0.006 to 0.53, <0.001), excluding suicide neutral, and were buffered by smaller positive associations at adjacent time lags. These results broadly indicate that as mobility decreased (i.e. more time at home), search volumes for terms within these categories increased at the same time. Similarly, for proportion of devices completely at home nationally, smaller positive cross-correlation coefficients were found at the -9-week lag for suicide seeking, mood and anxiety, and suicide prevention search volumes (0.28, 0.041 to 0.35, 0.01); however, these findings were flanked by smaller negative cross-correlation coefficients which obfuscated trend interpretation. Despite differences across mobility indicators, these results suggest the declining mobility prompted increased searches for mental health and suicide-related terms during the pandemic.

thumbnail
Fig 5. Heatmaps of cross-correlation coefficients for mobility indicators and Google Health Trends search volumes for suicide seeking, mood and anxiety, and social stressor terms nationally and in the NYC DMA: 2020–2021.

P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.

https://doi.org/10.1371/journal.pone.0260931.g005

Within the NYC DMA, however, there were few discernable patterns in cross-correlation coefficients for mental health and suicide-related terms across mobility indicators. Larger cross-correlation coefficients were irregularly distributed across negative and positive weekly lags and vacillated in direction. For time at home, only suicide seeking terms exhibited larger cross-correlation coefficients which alternated between positive and negative coefficients at weekly lags -3,-2, and 0 (-0.35, 0.009 to 0.42, 0.002). An analogous pattern also arose for suicide seeking search terms with proportion of devices completely at home at the -7 (-0.41, 0.002) and -6-week lags (0.3, 0.024). These alternating findings denote an inexplicable and potentially immaterial relationship between mobility and searches for suicide-seeking terms in the NYC DMA.

Sensitivity analyses

S3 Fig includes all heatmaps for sensitivity analyses with an extended study period of 2019 through 2021. As in primary analyses, we found larger, predominantly positive cross-correlation coefficients between economic stressor search volumes and mobility indicators across locations (0.21, 0.028 to 0.51, <0.001). These findings, however, spanned negative and positive weekly lags, signaling a bidirectional relationship between economic stress and mobility during this period.

Cross-correlation coefficients in sensitivity analyses were smaller and somewhat matched trends in primary analyses for mental health and suicide-related terms. Nationally, relatively larger positive cross-correlation coefficients for all search term categories except for suicide neutral and suicide seeking occurred at the -1-week lag between proportions of devices completely at home (0.19, 0.045 to 0.28, 0.004). As in primary analyses, however, these positive coefficients were immediately followed by smaller negative coefficients at the 0-week lag. Similarly, a string of negative cross-correlation coefficients for time at home at a 4-week lag were bound by positive correlations at adjacent lags. Ultimately, relationships, were mixed within term categories, especially between suicide seeking terms and time at home. Within the NYC DMA, findings were similarly ambiguous with no clear trends in cross-correlation coefficients.

Overall, the majority of analyses revealed positive concurrent associations between restricted mobility and search volumes for economic stressors. From 2020 through 2021 nationally, positive relationships were found between mobility restriction and mental health and suicide-related search volumes, excluding suicide neutral, and were strongest at the 0-week lag. Similar trends were not apparent within the NYC DMA.

Discussion

We employed mobility data documenting the proportion of cellular devices at home and time at home during the COVID-19 pandemic in the US and in New York City to assess whether restricted movement was associated with economic stress, mental health, and suicide as measured by Google Health Trends search volumes. Our research yielded two main findings. First, we found that pandemic mobility restrictions were contemporaneously linked to trajectories of online searches for terms related to economic distress across locations. Second, we found positive associations between declining mobility and mental health and suicide-related search volumes nationally during the pandemic. These results underscore the pervasiveness of pandemic-induced economic hardship and prompt further consideration of the relationships between isolation and mental health and suicide risk.

While Google data on mental health and suicide-related search volumes have been used as interim proxies for delayed epidemiological data during the pandemic [26, 4555], to our knowledge, no studies have assessed a time-variant indicator of population behavior, such as mobility, as an exposure. Cellular mobility data provide real-time information on actual mobility restrictions [33, 34], a likely important driver of pandemic-related negative mental health outcomes [74, 75]. We found that mobility declines were most notable during the early spring of 2020 when NYC became a global center of the pandemic [76] and stay-at-home orders and social distancing recommendations were first established [77]. We also found that search volumes for economic stressor terms rapidly increased during the early stages of the pandemic, in accordance with other studies [26, 49], and in line with evidence documenting high rates of unemployment and financial hardship in response to pandemic-related economic shut-downs [78, 79]. Moreover, declining mobility was strongly correlated with concurrent and future rises in search volumes for economic stressor terms across locations, especially from 2020 through 2021, confirming our hypothesis that mobility restrictions due to travel limitations and quarantine orders would predict economic distress due to consequent economic adversity and job loss.

Having established a proof of principle, we found that search volumes for mental health and suicide-related terms declined slightly during the pandemic onset in early 2020, matching existing research [26, 4651], and that mobility restrictions during January 2020 through January 2021 were associated with increased search volumes for these terms nationally, but not in the NYC DMA. Nationally, we identified strong concurrent relationships between mental health and suicide-related search volumes with time at home, which may be a more sensitive indicator of pandemic-induced isolation as it relates to declining psychiatric wellbeing. These findings align with longitudinal and cross-sectional studies in the US indicating that the prevalence of mental disorders and psychological distress increased during the early stages of the pandemic [18]. Findings of smaller correlations at heterogenous time lags that dissipated in geographically specific analyses and with the inclusion of pre-pandemic data are also consistent with evidence that lockdowns themselves have minor or mixed effects on population mental health [13, 80]. Further, provisional CDC estimates indicate that age-adjusted suicide rates during the first half of 2020 declined from those in 2019 [23]; however, emergency department visits related to suicide attempts for girls significantly increased over the course of the pandemic into 2021 [81], possibly explaining the inconsistent relationship between pandemic mobility restrictions and suicide seeking search volumes.

Foremost, our findings for economic stressor search volumes clarify the importance of financial concerns during the pandemic and highlight an opportunity to help those potentially vulnerable to medium- and long-term deteriorating mental health and increasing suicidal risk [31, 82]. Robust social safety net policies can mitigate the harms of prolonged financial distress [83, 84] and may contribute to ameliorating suicide risk [19]. Struggling individuals psychologically benefited from additional funds issued by the national government during the pandemic [85, 86] and, given the political will, governments can continue to grow and fortify the social safety net in the US [78].

The pandemic has presented many challenges to psychiatric epidemiology data collection, especially when administrative clinical service data and mortality records are lagged in their release. Our findings also contribute to the ongoing discussion surrounding the utility of Google search volumes in estimating mental health and suicide-related outcomes. Results for economic stressor, mental health, and, to some extent, suicide seeking terms matched trends in survey data. Yet attempts to assess the validity of Google search volumes, particularly in forecasting suicide deaths, [3641] have concluded that search volumes for suicide-related terms, even when surveying granular term categories, are noisy and inconsistent indicators of suicide rates. Knipe et al. (2021, preprint) [45] analyzed the relationship between search volumes for mental health terms and mental health outcomes in the United Kingdom during the pandemic, also finding that the time series did not generally align. Ultimately, searches within the Google platform, with over 267 million unique users in the US [87], do not exactly measure population mental health outcomes and are not comparable with survey data, but instead may serve as a useful barometer for real-time population thoughts and concerns.

The extent to which Google search volumes are useful for mental health and suicide risk surveillance purposes remains unclear. For instance, search volumes for suicide-related terms often spike after a celebrity suicide death [37], which may indicate news story interest rather than increased psychological distress. Suicide deaths, however, also increase after a celebrity dies by suicide [88]. This is an example of the “Werther effect” which postulates that such increases in emulative or imitative suicide deaths among the public result from heightened news media coverage and personal identification with the celebrity [89]. Notably, Google search volumes cannot be disaggregated by social groups, such as race, although minority racial groups emerged as at growing risk of suicide mortality during the early phases of the pandemic [90, 91]. Yet despite their coarseness, spikes in search volumes for suicide-related terms may serve as early indicators of actual increases in local suicidal behavior risk, making continued surveillance warranted for public health efforts.

This study included several limitations; foremost that both cellular mobility and GHT search volumes are imperfect proxies for actual population behavior and mental health. Notably, other factors, including rates of COVID-19 illness, hospitalizations, and deaths or heightened attention to mass media, may be strongly related to mental health and suicide-related search volumes and should be explored in future analyses. Further, GHT search volumes could not be stratified by key at-risk demographic groups or adjusted for other potentially contemporaneous events that may have affected time series. We also may have been able to model and assess long-term trends and seasonality more rigorously if protracted pre-pandemic mobility data were available. Due to a lack of contemporaneous suicide mortality data, we were unable to assess the validity of suicide-related search volumes and our results may not be generalizable beyond the US.

In conclusion, we found that GHT search volumes captured real-time increases in economic distress that were temporally linked to COVID-19 pandemic mobility restrictions in the US and in New York City. Associations were also found between declining mobility and rising mental health and suicide-related search volumes nationally. Our results align with pandemic-period research exploring Google search volumes in the US and available data on unemployment, mental health, and suicide risk. As epidemiological studies strive to keep pace with the psychological impacts of the pandemic, we believe that Google search volumes may be a useful interim data source for monitoring population mental health when carefully considered.

Supporting information

S1 Fig. Google Health Trends search volumes for suicide neutral, suicide prevention, and psychosis terms over time nationally and in the NYC DMA: 2020–2021.

https://doi.org/10.1371/journal.pone.0260931.s001

(TIF)

S2 Fig. Heatmaps of cross-correlation coefficients for mobility indicators and Google Health Trends search volumes for suicide neutral, suicide prevention, and psychosis terms nationally and in the NYC DMA: 2020–2021.

P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.

https://doi.org/10.1371/journal.pone.0260931.s002

(TIF)

S3 Fig. Heatmaps of cross-correlation coefficients for mobility indicators and Google Health Trends search volumes for all search term categories nationally and in the NYC DMA: 2019–2021.

P-values listed within cells. Note on interpretation: A cross-correlation coefficient at a negative weekly lag indicates that changes in the explanatory time series lead changes in the dependent time series that number of weeks later. A cross-correlation coefficient at a positive weekly lag indicates that changes in the dependent time series lead changes in the explanatory time series that number of weeks later. Correlations at the 0-week lag suggest that changes in both time series were concurrent. A positive cross-correlation coefficient indicates that there is a direct correlation between the time series and a negative cross-correlation coefficient indicates that there is an inverse correlation between the time series.

https://doi.org/10.1371/journal.pone.0260931.s003

(TIF)

S1 Table. Google search terms sourced from Lee (2020) [41].

https://doi.org/10.1371/journal.pone.0260931.s004

(PDF)

S2 Table. Counties matched to the New York City designated market area.

FIPS = Federal Information Processing Standards.

https://doi.org/10.1371/journal.pone.0260931.s005

(PDF)

S3 Table. Cross-correlation coefficients and p-values for mobility indicators Google Health Trends search volumes for all search term categories at weekly lags: 2020–2021.

P-value in parenthesis.

https://doi.org/10.1371/journal.pone.0260931.s006

(PDF)

S4 Table. Cross-correlation coefficients and p-values for mobility indicators and Google Health Trends search volumes for all search term categories at weekly lags: 2019–2021.

P-value in parenthesis.

https://doi.org/10.1371/journal.pone.0260931.s007

(PDF)

S5 Table. Fitted model parameters to mobility indicators nationally and in the NYC DMA.

ARIMA = autoregressive integrated moving average model. Non-seasonal ARIMA parameters = (p,d,q). p = autoregressive model order, d = degree of differencing, q = moving average model order.

https://doi.org/10.1371/journal.pone.0260931.s008

(PDF)

Acknowledgments

Mobility data was provided by Safe Graph, a data company that aggregates anonymized location data from numerous applications in order to provide insights about physical places, via the Placekey Community. To enhance privacy, SafeGraph excludes census block group information if fewer than five devices visited an establishment in a month from a given census block group.

References

  1. 1. Wu T, Jia X, Shi H, Niu J, Yin X, Xie J, et al. Prevalence of mental health problems during the COVID-19 pandemic: A systematic review and meta-analysis. Journal of Affective Disorders. 2021. pp. 91–98. pmid:33310451
  2. 2. Daly M, Sutin AR, Robinson E. Depression reported by US adults in 2017–2018 and March and April 2020. J Affect Disord. 2021;278: 131–135. pmid:32956962
  3. 3. Bueno-Notivol J, Gracia-García P, Olaya B, Lasheras I, López-Antón R, Santabárbara J. Prevalence of depression during the COVID-19 outbreak: A meta-analysis of community-based studies. Int J Clin Heal Psychol. 2021;21. pmid:32904715
  4. 4. Riehm KE, Holingue C, Smail EJ, Kapteyn A, Bennett D, Thrul J, et al. Trajectories of mental distress among U.S. adults during the COVID-19 pandemic. Ann Behav Med. 2021. pmid:33555336
  5. 5. Ettman CK, Abdalla SM, Cohen GH, Sampson L, Vivier PM, Galea S. Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic. JAMA Netw open. 2020;3: e2019686. pmid:32876685
  6. 6. Czeisler MÉ, Lane RI, Petrosky E, Wiley JF, Christensen A, Njai R, et al. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic—United States, June 24–30, 2020. MMWR Morb Mortal Wkly Rep. 2020;69: 1049–1057. pmid:32790653
  7. 7. Holingue C, Kalb LG, Riehm KE, Bennett D, Kapteyn A, Veldhuis CB, et al. Mental distress in the United States at the beginning of the COVID-19 pandemic. Am J Public Health. 2020;110: 1628–1634. pmid:32941066
  8. 8. Aknin L, Neve J-E De, Dunn E, Fancourt D, Goldberg E, Helliwell J, et al. Mental health during the first year of the COVID-19 pandemic: A review and recommendations for moving forward. PsyArXiv [Preprint]. 2021.
  9. 9. Shear MK, Ghesquiere A, Glickman K. Bereavement and complicated grief. Curr Psychiatry Rep. 2013;15: 406. pmid:24068457
  10. 10. Guldin MB, Ina Siegismund Kjaersgaard M, Fenger-Grøn M, Thorlund Parner E, Li J, Prior A, et al. Risk of suicide, deliberate self-harm and psychiatric illness after the loss of a close relative: A nationwide cohort study. World Psychiatry. 2017;16: 193–199. pmid:28498584
  11. 11. Hare-Duke L, Dening T, de Oliveira D, Milner K, Slade M. Conceptual framework for social connectedness in mental disorders: Systematic review and narrative synthesis. J Affect Disord. 2019;245: 188–199. pmid:30396057
  12. 12. Stickley A, Koyanagi A. Loneliness, common mental disorders and suicidal behavior: Findings from a general population survey. J Affect Disord. 2016;197: 81–87. pmid:26971125
  13. 13. Prati G, Mancini AD. The psychological impact of COVID-19 pandemic lockdowns: A review and meta-analysis of longitudinal studies and natural experiments. Psychol Med. 2021;51: 201–211. pmid:33436130
  14. 14. Hawton K, Lascelles K, Brand F, Casey D, Bale L, Ness J, et al. Self-harm and the COVID-19 pandemic: A study of factors contributing to self-harm during lockdown restrictions. J Psychiatr Res. 2021;137: 437–443. pmid:33774538
  15. 15. Holland KM, Jones C, Vivolo-Kantor AM, Idaikkadar N, Zwald M, Hoots B, et al. Trends in US emergency department visits for mental health, overdose, and violence outcomes before and during the COVID-19 pandemic. JAMA Psychiatry. 2021;78: 372–379. pmid:33533876
  16. 16. Harmon KJ, Fliss MD, Marshall SW, Peticolas K, Proescholdbell SK, Waller AE. The impact of the COVID-19 pandemic on the utilization of emergency department services for the treatment of injuries. Am J Emerg Med. 2021;47: 187–191. pmid:33892334
  17. 17. Walsh K, McLaughlin KA, Hamilton A, Keyes KM. Trauma exposure, incident psychiatric disorders, and disorder transitions in a longitudinal population representative sample. J Psychiatr Res. 2017;92: 212–218. pmid:28531836
  18. 18. MacIsaac MB, Bugeja LC, Jelinek GA. The association between exposure to interpersonal violence and suicide among women: a systematic review. Aust N Z J Public Health. 2017;41: 61–69. pmid:27774704
  19. 19. Milner A, Page A, LaMontagne AD. Long-term unemployment and suicide: A systematic review and meta-analysis. PLoS One. 2013;8: e51333. pmid:23341881
  20. 20. Nordt C, Warnke I, Seifritz E, Kawohl W. Modelling suicide and unemployment: A longitudinal analysis covering 63 countries, 2000–11. The Lancet Psychiatry. 2015;2: 239–245. pmid:26359902
  21. 21. Taquet M, Geddes JR, Husain M, Luciano S, Harrison PJ. 6-month neurological and psychiatric outcomes in 236 379 survivors of COVID-19: A retrospective cohort study using electronic health records. The Lancet Psychiatry. 2021;8: 416–427. pmid:33836148
  22. 22. Mazza MG, De Lorenzo R, Conte C, Poletti S, Vai B, Bollettini I, et al. Anxiety and depression in COVID-19 survivors: Role of inflammatory and clinical predictors. Brain Behav Immun. 2020;89: 594–600. pmid:32738287
  23. 23. Ahmand FB, Cisewski JA. Quarterly provisional estimates for selected indicators of mortality, 2018-Quarter 3 2020. National Center for Health Statistics. National Vital Statistics System, Vital Statistics Rapid Release Program; 2021.
  24. 24. Pirkis J, John A, Shin S, Delpozo-Banos M, Arya V, Analuisa-Aguilar P, et al. Suicide trends in the early months of the COVID-19 pandemic: An interrupted time-series analysis of preliminary data from 21 countries. The Lancet Psychiatry. 2021;8: 579–588. pmid:33862016
  25. 25. John A, Okolie C, Eyles E, Webb RT, Schmidt L, McGuiness LA, et al. The impact of the COVID-19 pandemic on self-harm and suicidal behaviour: A living systematic review. F1000Research. 2020;9: 1097. pmid:33604025
  26. 26. Halford EA, Lake AM, Gould MS. Google searches for suicide and suicide risk factors in the early stages of the COVID-19 pandemic. PLoS One. 2020;15: e0236777. pmid:32706835
  27. 27. Reger MA, Stanley IH, Joiner TE. Suicide mortality and coronavirus disease 2019—A perfect storm? JAMA Psychiatry. 2020;77: 1093–1094. pmid:32275300
  28. 28. John A, Pirkis J, Gunnell D, Appleby L, Morrissey J. Trends in suicide during the COVID-19 pandemic. The BMJ. BMJ Publishing Group; 2020. https://doi.org/10.1136/bmj.m4352 pmid:33184048
  29. 29. Kawohl W, Nordt C. COVID-19, unemployment, and suicide. The Lancet Psychiatry. 2020;7: 389–390. pmid:32353269
  30. 30. Rogers JP, Chesney E, Oliver D, Begum N, Saini A, Wang S, et al. Suicide, self-harm and thoughts of suicide or self-harm in infectious disease epidemics: A systematic review and meta-analysis. Epidemiol Psychiatr Sci. 2021;30: e32. pmid:33902775
  31. 31. Karanikolos M, Heino P, McKee M, Stuckler D, Legido-Quigley H. Effects of the global financial crisis on health in high-income OECD countries: A narrative review. Int J Heal Serv. 2016;46: 208–240. pmid:27076651
  32. 32. Tanaka T, Okamoto S. Increase in suicide following an initial decline during the COVID-19 pandemic in Japan. Nat Hum Behav. 2021;5: 229–238. pmid:33452498
  33. 33. Lee M, Zhao J, Sun Q, Pan Y, Zhou W, Xiong C, et al. Human mobility trends during the early stage of the COVID-19 pandemic in the United States. PLoS One. 2020;15: e0241468. pmid:33166301
  34. 34. Xiong C, Hu S, Yang M, Younes H, Luo W, Ghader S, et al. Mobile device location data reveal human mobility response to state-level stay-at-home orders during the COVID-19 pandemic in the USA. J R Soc Interface. 2020;17: 20200344. pmid:33323055
  35. 35. Allcott H, Boxell L, Conway J, Gentzkow M, Thaler M, Yang D. Polarization and public health: Partisan differences in social distancing during the coronavirus pandemic. J Public Econ. 2020;191: 104254. pmid:32836504
  36. 36. Barros JM, Melia R, Francis K, Bogue J, O’Sullivan M, Young K, et al. The validity of Google Trends search volumes for behavioral forecasting of national suicide rates in Ireland. Int J Environ Res Public Health. 2019;16: 3201. pmid:31480718
  37. 37. Gunn Iii JF, Goldstein SE, Lester D. The impact of widely publicized suicides on search trends: Using Google Trends to test the Werther and Papageno effects. Arch Suicide Res. 2020;24: 142–155. pmid:30300114
  38. 38. Solano P, Ustulin M, Pizzorno E, Vichi M, Pompili M, Serafini G, et al. A Google-based approach for monitoring suicide risk. Psychiatry Res. 2016;246: 581–586. pmid:27837725
  39. 39. Akers RL, Jensen GF. The empirical status of social learning theory of crime and deviance: The past, present, and future. Taking stock: The status of criminological theory. Piscataway, NJ, US: Transaction Publishers; 2006. pp. 37–76.
  40. 40. Tran US, Andel R, Niederkrotenthaler T, Till B, Ajdacic-Gross V, Voracek M. Low validity of Google Trends for behavioral forecasting of national suicide rates. PLoS One. 2017;12: e0183149. pmid:28813490
  41. 41. Lee J-Y. Search trends preceding increases in suicide: A cross-correlation study of monthly Google search volume and suicide rate using transfer function models. J Affect Disord. 2020;262: 155–164. pmid:31733460
  42. 42. Cervellin G, Comelli I, Lippi G. Is Google Trends a reliable tool for digital epidemiology? Insights from different clinical settings. J Epidemiol Glob Health. 2017;7: 185. pmid:28756828
  43. 43. Rovetta A. Reliability of Google Trends: Analysis of the limits and potential of web infoveillance during COVID-19 pandemic and for future research. Front Res Metrics Anal. 2021;6. pmid:34113751
  44. 44. Dagher SH, Lamé G, Hubiche T, Ezzedine K, Duong TA. The influence of media coverage and governmental policies on Google queries related to COVID-19 cutaneous symptoms: Infodemiology study. JMIR Public Heal Surveill. 2021;7. pmid:33513563
  45. 45. Knipe D, Gunnell D, Evans H, John A, Fancourt D. Is Google Trends a useful tool for tracking mental and social distress during a public health emergency? A time-series analysis. medRxiv [Preprint]. 2021; 2021.02.18.21251966. pmid:34348169
  46. 46. Brodeur A, Clark AE, Fleche S, Powdthavee N. COVID-19, lockdowns and well-being: Evidence from Google Trends. J Public Econ. 2021;193: 104346. pmid:33281237
  47. 47. Jacobson NC, Lekkas D, Price G, Heinz M V, Song M, O’Malley AJ, et al. Flattening the mental health curve: COVID-19 stay-at-home orders Are associated with alterations in mental health search behavior in the United States. JMIR Ment Jealth. 2020;7: e19347–e19347. pmid:32459186
  48. 48. Sinyor M, Spittal MJ, Niederkrotenthaler T. Changes in suicide and resilience-related Google searches during the early stages of the COVID-19 pandemic. Canadian Journal of Psychiatry. 2020. pp. 741–743. pmid:32524848
  49. 49. Knipe D, Evans H, Marchant A, Gunnell D, John A. Mapping population mental health concerns related to COVID-19 and the consequences of physical distancing: a Google trends analysis. Wellcome Open Res. 2020;5: 82. pmid:32671230
  50. 50. Misiak B, Szcześniak D, Koczanowicz L, Rymaszewska J. The COVID-19 outbreak and Google searches: Is it really the time to worry about global mental health? Brain Behav Immun. 2020;87: 126–127. pmid:32360605
  51. 51. Hoerger M, Alonzi S, Perry LM, Voss HM, Easwar S, Gerhart JI. Impact of the COVID-19 pandemic on mental health: Real-time surveillance using Google Trends. Psychol Trauma Theory, Res Pract Policy. 2020;12: 567–568. pmid:32790441
  52. 52. Chen H, Zhang K. Insight into the psychological problems on the epidemic of COVID-19 in China by online searching behaviors. J Affect Disord. 2020;276: 1093–1094. pmid:32771861
  53. 53. Tennant P, Gonzalez J, Rowan M, Hilbelink C, Moore Q. The association between new COVID-19 cases and Google searches for mental health. Adv [Preprint]. 2020. https://doi.org/10.31124/advance.12375020.v2
  54. 54. Ayers JW, Leas EC, Johnson DC, Poliak A, Althouse BM, Dredze M, et al. Internet searches for acute anxiety during the early stages of the COVID-19 pandemic. JAMA Intern Med. 2020. pmid:32832984
  55. 55. Lin Y-H, Chiang T-W, Lin Y-L. Increased internet searches for insomnia as an indicator of global mental health during the COVID-19 pandemic: Multinational longitudinal study. J Med Internet Res. 2020;22: e22181. pmid:32924951
  56. 56. Nischan U. Uneven impact: What job loss during COVID-19 means for New Yorkers now and into the future. Dec 2020 [cited 23 Mar 2021]. Available: https://www1.nyc.gov/assets/dca/downloads/pdf/partners/Uneven_Impact.pdf
  57. 57. Wadhera RK, Wadhera P, Gaba P, Figueroa JF, Joynt Maddox KE, Yeh RW, et al. Variation in COVID-19 hospitalizations and deaths across New York City boroughs. JAMA. American Medical Association; 2020. pp. 2192–2195. pmid:32347898
  58. 58. Borjas GJ. Business closures, stay-at-home restrictions, and COVID-19 testing outcomes in New York City. Prev Chronic Dis. 2020;17. pmid:32945766
  59. 59. Taylor DB. A timeline of the Coronavirus pandemic. In: The New York Times [Internet]. 17 Mar 2021 [cited 16 Jun 2021]. Available: https://www.nytimes.com/article/coronavirus-timeline.html
  60. 60. Box G, Jenkins G. Time series analysis: Forecasting and control. San Francisco: Holden-Day; 1970.
  61. 61. Cryer JD, Chan K-S. Time series analysis: With applications in R. New York: Springer; 2008. https://doi.org/10.1007/978-0-387-75959-3
  62. 62. Stocking G, Matsa KE. Using Google Trends data for research? Here are 6 questions to ask. In: Medium [Internet]. 27 Apr 2017 [cited 11 Jun 2021]. Available: https://medium.com/@pewresearch/using-google-trends-data-for-research-here-are-6-questions-to-ask-a7097f5fb526
  63. 63. Nuti S V., Wayda B, Ranasinghe I, Wang S, Dreyer RP, Chen SI, et al. The use of Google Trends in health care research: A systematic review. PLoS One. 2014;9. pmid:25337815
  64. 64. DMA Regions. In: Nielson [Internet]. 2021 [cited 24 Mar 2021]. Available: https://www.nielsen.com/us/en/intl-campaigns/dma-maps/
  65. 65. SafeGraph. Social Distancing Metrics. 2021. Available: https://docs.safegraph.com/docs/social-distancing-metrics
  66. 66. Lamb MR, Kandula S, Shaman J. Differential COVID-19 case positivity in New York City neighborhoods: Socioeconomic factors and mobility. Influenza Other Respi Viruses. 2021;15: 209–217. pmid:33280263
  67. 67. Hyndman RJ, Khandakar Y. Automatic time series forecasting: The forecast package for R. J Stat Softw. 2008;27: 1–22.
  68. 68. Derrick T, Thomas J. Time series analysis: The cross-correlation function. Innovative Analyses of Human Movement. 2004. pp. 189–205.
  69. 69. Schober P, Schwarte LA. Correlation coefficients: Appropriate use and interpretation. Anesth Analg. 2018;126: 1763–1768. pmid:29481436
  70. 70. Cohen J. A power primer. Psychol Bull. 1992;112: 155–159. pmid:19565683
  71. 71. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, et al. Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. Eur J Epidemiol. 2016;31: 337. pmid:27209009
  72. 72. Amrhein V, Korner-Nievergelt F, Roth T. The earth is flat (p > 0.05): Significance thresholds and the crisis of unreplicable research. PeerJ. 2017;5. pmid:28698825
  73. 73. R Core Team. R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing; 2021.
  74. 74. Pantelaki E, Maggi E, Crotti D. Mobility impact and well-being in later life: A multidisciplinary systematic review. Res Transp Econ. 2020;86: 100975.
  75. 75. Polku H, Mikkola TM, Portegijs E, Rantakokko M, Kokko K, Kauppinen M, et al. Life-space mobility and dimensions of depressive symptoms among community-dwelling older adults. Aging Ment Heal. 2015;19: 781–789. pmid:25376479
  76. 76. McKinley J. New York City region is now an epicenter of the Coronavirus pandemic. In: The New York Times [Internet]. 22 Mar 2020 [cited 11 May 2021]. Available: https://www.nytimes.com/2020/03/22/nyregion/Coronavirus-new-York-epicenter.html
  77. 77. Mervosh S, Lu D, Swales V. See which states and cities have told residents to stay at home. In: The New York Times [Internet]. 20 Apr 2020 [cited 11 May 2021]. Available: https://www.nytimes.com/interactive/2020/us/coronavirus-stay-at-home-order.html
  78. 78. Moffitt RA, Ziliak JP. COVID-19 and the US safety net. Fisc Stud. 2020;41: 515–548. pmid:33362310
  79. 79. Parker K, Minkin R, Bennett J. Economic fallout from COVID-19 continues to hit lower-income Americans the hardest. In: Pew Research Center [Internet]. 24 Sep 2020 [cited 11 May 2021]. Available: https://www.pewresearch.org/social-trends/2020/09/24/economic-fallout-from-covid-19-continues-to-hit-lower-income-americans-the-hardest/
  80. 80. Lau JTF, Yang X, Tsui HY, Pang E, Wing YK. Positive mental health-related impacts of the SARS epidemic on the general public in Hong Kong and their associations with other negative impacts. J Infect. 2006;53: 114–124. pmid:16343636
  81. 81. Yard E, Radhakrishnan L, Ballesteros MF, Sheppard M, Gates A, Stein Z, et al. Emergency department visits for suspected suicide attempts among persons aged 12–25 years before and during the COVID-19 pandemic—United States, January 2019–May 2021. MMWR Morb Mortal Wkly Rep. 2021;70. pmid:34138833
  82. 82. Knapp EA, Bilal U, Dean LT, Lazo M, Celentano DD. Economic insecurity and deaths of despair in US counties. Am J Epidemiol. 2019;188: 2131–2139. pmid:31172197
  83. 83. Cylus J, Glymour MM, Avendano M. Do generous unemployment benefit programs reduce suicide rates? A state fixed-effect analysis covering 1968–2008. Am J Epidemiol. 2014;180: 45–52. pmid:24939978
  84. 84. Cylus J, Glymour MM, Avendano M. Health effects of unemployment benefit program generosity. Am J Public Health. 2015;105: 317–323. pmid:25521897
  85. 85. Berkowitz SA, Basu S. Unmet social needs and worse mental health after expiration of COVID-19 Federal Pandemic Unemployment Compensation. Health Aff (Millwood). 2021;40: 426–434. pmid:33600235
  86. 86. Berkowitz SA, Basu S. Unemployment insurance, health-related social needs, health care access, and mental health during the COVID-19 pandemic. JAMA Intern Med. 2020;181: 699–702. pmid:33252615
  87. 87. Johnson J. Google—Statistics & Facts. In: Statista [Internet]. 25 Feb 2021 [cited 11 May 2021]. Available: https://www.statista.com/topics/1001/google/#dossierSummary
  88. 88. Fink DS, Santaella-Tenorio J, Keyes KM. Increase in suicides the months after the death of Robin Williams in the US. PLoS One. 2018;13. pmid:29415016
  89. 89. Phillips DP. The influence of suggestion on suicide: Substantive and theroretical implications of the Werther effect. Am Sociol Rev. 1974;39: 340–354. pmid:4844994
  90. 90. Mitchell TO, Li L. State-level data on suicide mortality during COVID-19 quarantine: Early evidence of a disproportionate impact on racial minorities. Psychiatry Res. 2021;295: 113629. pmid:33290944
  91. 91. Bray MJC, Daneshvari NO, Radhakrishnan I, Cubbage J, Eagle M, Southall P, et al. Racial differences in statewide suicide mortality trends in Maryland during the Coronavirus disease 2019 (COVID-19) pandemic. JAMA Psychiatry. 2020. pmid:33325985