Your privacy, your choice

We use essential cookies to make sure the site can function. We also use optional cookies for advertising, personalisation of content, usage analysis, and social media.

By accepting optional cookies, you consent to the processing of your personal data - including transfers to third parties. Some third parties are outside of the European Economic Area, with varying standards of data protection.

See our privacy policy for more information on the use of your personal data.

for further information and to change your choices.
Skip to main content
Download PDF
Download ePub

Why do International Health Regulations self-assessment capacities (SPAR) scores not predict COVID-19 control outcomes? – analysis of the relationship between SPAR scores and COVID-19 resilience scores in 2021

Globalization and Health volume 21, Article number: 19 (2025) Cite this article

Abstract

Introduction

This study analyses the relationship between SPAR and the COVID-19 resilience score (CRS) in 80 countries in 2021 to achieve its objective.

Methods

We adopted the concept of Bloomberg’s COVID Resilience Ranking to form the CRS, which encompasses three indicators: Reopening Status, COVID Status, and Quality of Life. The average scores of 13 SPAR capacities focused on infectious disease control in 2021 were calculated. Paired t-tests were applied to evaluate the significance of monthly changes in countries’ CRSs. Then, we conducted univariate and multivariate linear regressions to examine the relationship between the SPAR and CRS scores and each CRS indicator.

Results

The CRS in 80 countries fluctuated throughout 2021. Linear regression revealed a significant relationship between countries’ SPAR scores and CRS (B = 0.03, 95% CI = 0.001, 0.06). Among the CRS indicators, the SPAR was significantly associated with only the Quality of Life indicator (B = 0.01, 95% CI = 0.002, 1.52) and not the Reopening Status and COVID Status indicators. An increase in SPAR score, along with an increase in Governmental Effectiveness, was associated with increased CRS (Adjusted R2 = 0.52, p < 0.05). Moreover, an increase in countries’ SPAR scores was significantly linked to an improvement in people’s Quality of Life (Adjusted R2 = 0.37, p < 0.05).

Conclusion

The significant relationship between the SPAR and COVID Resilience Scores, particularly the Quality of Life indicator indicates that the lack of accuracy in the ability of the SPAR score to predict COVID-19 control outcomes is attributed to the reliance of the measurement solely on the disease perspective and the limited inclusion of social aspects in the SPAR capacity assessments.

Clinical trial number

Not Applicable

Introduction

The International Health Regulations 2005 (IHR 2005) mandate that member states establish minimum core capacities for pandemic preparedness as an important approach to enhancing global health security [1, 2]. To assess the progress of capacity development, the World Health Organization (WHO) requires member states to annually conduct a self-evaluation of their IHR core capacities, known as the State Party Annual Reporting (SPAR) [1, 2]. Since its implementation in 2007, there has been a gradual increase in member states’ engagement with the IHR [3]. This heightened participation has been demonstrated to aid countries in enhancing their capacity for pandemic preparedness [4,5,6].

Prior to the COVID-19 pandemic, studies [4, 5] found that this approach was effective and accountable for supporting member states to enhance their ability to address significant public health threats. A study [5] indicated that a 10% increase in SPAR score was associated with a notable decrease of 14–20% in the incidence of transboundary Infectious Disease Threat Events (IDTEs) globally. Furthermore, another study revealed that countries with low SPAR scores face a significantly greater risk of experiencing poor infectious disease outcomes than countries with high average SPAR scores [4]. Based on the assumption that SPAR scores do reflect countries’ capacity, a study in the early phase of the COVID-19 pandemic [7] predicted that countries with higher SPAR scores would experience less severe impacts from the pandemic. However, throughout the COVID-19 pandemic, particularly in the first year, countries with both low and high SPAR scores experienced severe consequences [7, 8].

The devastating impact of the COVID-19 pandemic resulted in it being characterized as a global disaster [9]. Furthermore, in addition to its profound effects on physical health, the pandemic had detrimental effects on the mental and social well-being of most of the global population [10, 11]. As of December 2022, there were more than 600 million confirmed cases of COVID-19 and more than 6 million deaths [11]. Before the pandemic was declared ended on May 5, 2023, it was estimated that by early 2025, COVID-19 would have caused at least 7 million deaths [12]. In addition to health loss [10], reports indicate that the pandemic led to increased inequality, exclusion, discrimination, and unemployment [13]. The COVID-19 pandemic triggered the most severe economic recession since World War II [14], placing significant strain on societies [10, 13, 15]. Such a broad and severe impact of COVID-19 led to concern about countries’ resilience during the pandemic and what it might tell us about preparation for future pandemics.

The United Nations International Strategy for Disaster Reduction (UNISDR) defines resilience as “the ability of an exposed system, community, or society to adapt, bounce back, and recover from the impact of stressors by considering multiple elements, including social, human, physical, financial, and natural resources, in a timely and efficient manner” [16]. This definition emphasizes the significance of considering these elements from a wider perspective, particularly from a social standpoint.

Recognizing the COVID-19 pandemic as a significant disaster that necessitates a focus on countries’ resilience capacity, Bloomberg introduced the COVID Resilience Ranking [17] in November 2020. This ranking system aims to assess countries’ performance in managing the pandemic by evaluating their ability to strike a balance between pandemic control measures and the maintenance of people’s quality of life. Countries with higher resilience rankings demonstrate greater ability to effectively control the spread of the virus while minimizing social and economic disruptions [17]. In contrast to the IHR SPAR score, which mainly focuses on countries’ capacity to detect, assess, and analyze public health epidemics [15], the concept of resilience during COVID-19 expanded from focusing only on health indicators to broader social factors such as Reopening Status and Quality of Life. Throughout the COVID-19 pandemic from the beginning of 2020 to 2023, the continuous debate regarding the policy balance between strict public health measures and economic development and people’s freedom of movement evidenced the importance of involving social concerns in countries’ capacity to respond to the pandemic. Since COVID-19 has been prevalent for over 4 years, much longer than previous pandemics such as SARS, which ended within 6 months, the need for countries to be able to handle the balance between disease control and social concerns has increased. However, the current SPAR tool does not cover such perspectives that might help countries better prepare for the next pandemic.

Since the effects of COVID-19 extended beyond physical health to encompass mental and social well-being [10, 11], it is plausible that the inability of SPAR scores to accurately predict disease control outcomes is due to the lack of ability to balance health measures with social concerns, especially as the pandemic progressed from short-term to medium- and long-term effects.

Therefore, we conducted this study to evaluate the associations between SPAR scores and countries’ resilience during the COVID-19 pandemic. Since resilience is a broader capacity that covers countries’ capacity for health control, SPAR scores should be associated with resilience. Furthermore, with respect to the ability of the IHR to control infectious disease, theoretically, SPAR scores should be significantly associated with health indicators related to resilience. However, because the SPAR does not predict disease control outcomes due to the inability to balance health measures with social concerns, we hypothesized that the SPAR is associated with factors other than health.

Methods

This ecological study involved an analysis of a total of 80 countries. The inclusion criteria for countries were based on the availability of complete datasets for all variables employed in this study. The focus of this study was to evaluate the COVID Resilience Scores (CRS), a dependent variable, in relation to countries’ SPAR scores related to infectious disease control, serving as the independent variable. Additionally, referring to the Systemic Rapid Assessment (SYSRA) Toolkit [18], we accounted for two confounding variables in this study: Civil Liberties (CL) and Government Effectiveness (GE).

The analysis focused on the second year of the COVID-19 pandemic (2021), a critical phase in which countries had moved beyond the initial emergency response and were transitioning into long-term pandemic management. By this time, vaccination campaigns were underway, public health measures had evolved, and governments were balancing disease control with economic and social considerations.

By examining resilience factors in 2021, this study provides valuable insights into how countries adapted to ongoing challenges, refined their pandemic responses, and integrated public health measures with societal well-being which remain relevant for strengthening preparedness strategies against future health crises.

In this study, certain numerical data variables were transformed into binary categories. This conversion aimed to simplify the description of countries’ characteristics, facilitate scoring, and improve interpretability. The statistical significance threshold was established at p < 0.05. All analytical procedures were conducted using R version 4.3.1.

Countries’ COVID resilience scores (CRS)

Countries’ resilience capacity was assessed using the COVID Resilience Score (CRS), adapted from Bloomberg’s COVID Resilience Rankings [17]. In accordance with Bloomberg’s approach, the CRS has three measurement indicators (Reopening Status, COVID Status, and Quality of Life), each consisting of four measurement components. However, while the CRS retained the three Bloomberg indicators, some components were substituted in order to achieved the study aims (a comparison of indicators is provided in Supplementary Table 1s). In addition, unlike the Bloomberg approach, which analyzes only the fifty-three largest economies [17], CRS analyzes all countries with available data. By using these modified metrics, the CRS represents an attempt to provide a comprehensive assessment of countries’ resilience to the COVID-19 pandemic in 2021.

Under the CRS framework, equal weight proportions are assigned to all three indicators for calculating the CRS scores of these countries. A binary scale (0 or 1) was used to assess the twelve measurement components, and the average score was used as the cutoff point (the scoring system for CRS is provided in Supplementary Table 2s). Consequently, as the CRS encompasses twelve components, the CRS score for each country ranges from 0 (indicating the lowest score) to 12 (indicating the highest score).

Reopening status

The first indicator in the CRS is the Reopening Status, which aims to evaluate the degree to which countries are advancing toward a return to pre-COVID conditions from perspectives including social activities, economics, business, and tourism [17]. The Reopening Status metric consists of four distinct components: “People Covered by Vaccines” [19], “Lockdown Severity” [20], “Community Immobility” [21], and “Travel Reopening” [22].

“People Covered by Vaccines” [18] was defined as the total number of individuals who received at least one vaccine dose divided by the total population of the respective country. Moreover, “Lockdown Severity” [19] is defined as the stringency index, a composite measure based on nine response indicators encompassing factors such as school closures, workplace closures, and travel bans. This index is rescaled to a value ranging from 0 (indicating openness) to 100 (indicating full closure). The data for these components were sourced from the Our World in Data website.

The “Community Immobility” [20] component is defined as the alteration in the duration of time spent at places of residence relative to a baseline day. The data were collected from both Our World in Data and Google LLC. Moreover, “Travel Reopening” [21] is defined as travel restrictions implemented by governments and includes factors such as country travel restrictions, flight restrictions, the requirement of COVID-19 certificates, quarantine measures, and vaccination coverage. The data of this component were acquired from The Humanitarian Data Exchange (HDX), Office for the Coordination of Humanitarian Affairs (OCHA).

These components were selected to ensure a comprehensive evaluation of reopening progress, encompassing both international and domestic levels. Each individual component within the Reopening Status indicator was assigned a binary score of either 0 or 1 (Table 1s in Supplementary Table 2s). For the “People Covered by Vaccines” and “Travel Reopening” components, a score of 1 is assigned if the value exceeds the cutoff point, while a score of 0 is assigned for values below it. Conversely, for the “Lockdown Severity” and “Community Immobility” components, a score of 1 is given if the value falls below the cutoff point, and a score of 0 is assigned if the value is above it. Consequently, the cumulative score for the indicator of Reopening Status ranges from 0 (indicating a restricted status) to 4 (indicating a lenient status).

COVID status

The COVID Status indicator aims to determine the severity status of the COVID-19 pandemic in these countries. Four distinct components were utilized to measure this indicator: Reproduction Rate (RR), Case Fatality Rate (CFR), Test per case (TPC), and Positive test Rate (PTR). All of the data for these components were generated from Our World in Data Website [23]. These components were used in the CRS to better illustrate the COVID-19 status of the countries over a one-year timeframe.

The RR was defined as the average number of new infections caused by a single infected individual. The CFR represents the ratio of confirmed deaths to confirmed cases. TPC is the number of tests divided by the number of confirmed cases, while PTR is the number of confirmed cases divided by the number of tests, expressed as a percentage.

Among these 4 components, only the TPC is assigned a score of 1 if its value exceeds the cutoff point and a score of 0 if it is below the cutoff point. Moreover, for the other 3 components, RR, CFR, and PR, a score of 1 is given if their values fall below the cutoff point, and a score of 0 is assigned if their values are above the cutoff point. The total score of the COVID Status indicator for each country ranges from 0 (indicating a severe situation) to 4 (indicating a mild situation).

Quality of life

The last indicator of CRS is the Quality of Life. In order to assess the Quality of Life, several dimensions and individual perceptions were considered, reflecting the multidimensional nature of the concept, as defined by the WHO. The WHO defines quality of life as an individual’s perception of his or her position in life, considering his or her goals, expectations, standards, and concerns, within the cultural and values systems of his or her environment [24]. Thus, this study incorporates the following components within the Quality of Life indicator in the CRS framework: the Happiness Index [25], the Peace Index [26], annual GDP growth in 2021 [27], and Human Development Index (HDI) [28].

The Happiness Index provides insights into various aspects of well-being. It is a renowned survey that assesses the global happiness levels of countries based on citizens’ self-perceived well-being. This index not only ranks countries worldwide by subjective well-being but also delves into how social, urban, and natural environments collectively influence happiness, providing valuable insights into various dimensions of well-being. The data were sourced from the World Happiness Report 2021 [25].

Annual GDP growth offers an indication of economic performance and development. It is defined as the annual percentage growth rate of GDP at market prices based on constant local currency. These data serve as indicators of a country’s economic performance and development. For this study, we utilized GDP growth data from 2020 to 2021, which were obtained from The World Bank [27].

The Peace Index component’s data were generated from the Global Peace Index 2021 and obtained from the Vision of Humanity website [26]. Providing insights into societal peace and stability, the Peace Index is defined as a composite measure that assesses the peacefulness of countries; it comprises 23 quantitative and qualitative indicators, each weighted on a scale of 1 to 5, where a lower score indicates a higher level of peace.

The final component of the Quality of Life Indicator is the HDI. The HDI reflects the level of human development in each country by measuring three crucial dimensions of human development: a long and healthy life, access to education, and a decent standard of living [29]. Within this indicator, components with values exceeding the predefined cutoff point receive a score of 1, while those falling below the cutoff point receive a score of 0. Consequently, the Quality of Life indicator is assessed on a scale ranging from 0 (indicating poor quality of life) to 4 (indicating fair quality of life) for each country. The data for the HDI component [28] were sourced from the UNDP website.

Countries’ SPAR scores related to infectious disease control

This study used 2021 SPAR scores for the countries retrieved from the WHO website in January 2022 [30]. In 2021, 15 SPAR capacities were reported by the countries. However, since the focus of this study revolves around infectious disease control capacities, only 13 SPAR capacities directly related to infectious disease control were considered. Capacities related to “Chemical contamination” and “Radiation events” were excluded from the analysis.

The scores assigned to each SPAR capacity ranged from 20 (indicating the lowest score) to 100 (indicating the highest score). A score of 20 signifies that the particular capacity has not yet been established within a country, while a score of 100 indicates the sustainable implementation of the capacity. These scores were utilized to assess the level of infectious disease control capacity in each country, providing insights into people’s readiness and preparedness in combating infectious diseases.

Confounding variables

Referring to the Systemic Rapid Assessment (SYSRA) [18], this study considered Civil Liberties (CL) [31,32,33]. and Government Effectiveness (GE) [34] as confounding variables. CL data for this study were collected from the Freedom House website in 2021. It represents the transparency of countries in evaluating their national infectious disease capacity [35], with open communication and information contributing to public supervision and preventing manipulation of assessment outcomes. CL score is rated on a scale from 1 to 7, with 1 representing the highest degree of freedom and 7 the lowest. The score is derived from 15 civil liberties indicators, assessed through a set of specific questions. These indicators are categorized into four subcategories: Freedom of Expression and Belief (4 questions); Associational and Organizational Rights (3 questions); Rule of Law (4 questions); and Personal Autonomy and Individual Rights (4 questions).

Next, GE is a component derived from The Worldwide Governance Indicators (WGI), an extensive framework for assessing governance. Within the WGI, the GE specifically measures the quality of public services, policy formulation, and the implementation of government actions. GE scores range from − 2.50 (indicating the lowest GE) to 2.50 (indicating the highest GE) [34]. This study utilized 2021 GE data obtained from The World Bank website. The inclusion of GE in this study aligns with the findings of previous research that has underscored the pivotal role of good governance and the government’s capacity to prevent and control infectious diseases [36, 37].

Data analysis

To evaluate the monthly trend of countries’ CRS and its Reopening and COVID Status indicators, the monthly score of each indicator was calculated and is presented in Fig. 1. For the indicator with daily data, we first calculate the average value of each component from 80 countries every month. These values were subsequently used as cutoff points to classify countries into high- and low-performance groups, with scores of 0 or 1 every month. We then summed the scores of the components of each indicator to obtain the monthly score. This applies to both the Reopening Status and COVID Status. For the indicator with only yearly data, such as Quality of Life, we assumed that the values of each component within this indicator were the same every month throughout the year. We further determined the significance of the monthly changes for Reopening and COVID Status indicators by conducting paired t-tests.

Fig. 1
figure 1

Changes in COVID resilience scores and its indicators in 2021

Full size image

With the cutoff point as the average score, countries were divided into high-SPAR and low-SPAR groups for analysis. The chi-square test was used to compare countries’ characteristics including their income level and geographical region, between the different SPAR and CRS score groups. Then, we conducted univariate and multivariate linear regression analyses to develop Models 1 and 2 to evaluate the associations between countries’ SPAR scores and their CRS. While Model 1 sought to identify associations between countries’ infectious disease control capacities (measured by SPAR scores) and their CRS scores in 2021, Model 2 was developed to ensure internal validity and identify factors that contribute to countries’ CRS in the same year by incorporating confounding variables.

Results

Comparison of countries’ characteristics between different SPAR and CRS groups

A comparison of country characteristics between the different SPAR and CRS groups is shown in Table 1. Among the 80 countries analyzed, the majority were high-income countries (HICs) (40%), followed by high-middle-income countries (HMICs) (33.8%), low-middle-income countries (LMICs) (21.3%), and low-income countries (LICs) (5%). European countries constituted 35% of our analyzed countries, followed by Asian regions at 25%, American regions at 21.3%, and African regions at 18.7%. Regarding the CRS, only 3 countries (3.8%) had scores ranging from 9 to 12, while the scores of 6 countries (7.5%) ranged from 0 to 4. The majority of countries (88.7%) had scores ranging from 5 to 8. The geographic distribution of SPAR scores and CRS is uneven, as 63.6% of European countries have CRS scores ranging between 9 and 12, and 40.5% of countries with high SPAR scores are located in Europe. In addition, the chi-square test showed that (Table 1) income level, region, civil liberties and government effectiveness were not significantly different (p > 0.05) between the CRS groups. However, all of these countries’ characteristics were significantly different (p < 0.05) between the SPAR groups.

Table 1 Comparison of countries’ characteristics between CRS and SPAR score groups
Full size table

Changes in COVID resilience scores and its indicators in 2021

The monthly average CRS of the countries is shown in Fig. 1. The CRS exhibited fluctuations from January to December 2021 in 80 countries. The highest CRS scores were recorded in May (6.85 points) and November 2021 (7.15 points), while the lowest was recorded in January 2021 (6.14 points). Paired t-test analysis revealed significant increases in CRS scores (p < 0.05) in May (+ 0.47 point from April) and November (+ 0.99 point from October). Conversely, the CRS of these 80 countries significantly decreased (p < 0.05) in June (-0.55 point from May) and December (-0.46 point from November) of the same year.

The analysis of the Reopening Status indicator showed that the 80 countries experienced a significant shift toward leniency in May (2.41 points, p < 0.05), followed by a subsequent shift toward increased restrictions in June (1.99 points, p < 0.05). Furthermore, in comparison to those in the previous month, the COVID Status in these 80 countries were significantly milder in February (1.83 points, p < 0.05) and November 2021 (2.54 points, p < 0.05), while the COVID Status were more severe in August (1.75 points, p < 0.05) and December 2021 (2.26 points, p < 0.05).

Associations between countries’ SPAR and COVID resilience scores in 2021

The results from Model 1 (Table 2) revealed a significant association between the SPAR scores of countries and their CRS scores, including all three indicators. Specifically, an increase in SPAR scores was significantly linked to an increase in CRS (p < 0.05) and its three indicators (p < 0.05).

Model 2 (Table 3) confirmed the significant association between the SPAR scores of countries and their CRS scores. Model 2 revealed that the SPAR (B = 0.04, p < 0.01) and GE (B = 0.90, p < 0.01) were factors associated with CRS. An increase in SPAR scores and GE was significantly associated with an increase in CRS (adjusted R2 = 0.52, F = 29.11; p < 0.01).

Table 2 Association between countries’ SPAR and COVID resilience scores in 2021
Full size table
Table 3 Factors associated with countries’ COVID resilience scores in 2021
Full size table

Among the CRS indicators, the findings in Model 2 indicate that countries’ SPAR scores are significantly associated only with the Quality of Life indicator (adjusted R2 = 0.37). An increase in countries’ SPAR scores was significantly linked to an increase in their scores in Quality of Life indicator (B = 0.02, 95% CI = 0.006, 0.04).

Moreover, Model 2 revealed that none of the factors included in the model exhibited a significant association with the COVID Status indicator, while factors such as CL (B = 0.17, 95% CI = 0.06, 0.28) and GE (B = 0.45, 95% CI = 0.18, 0.72) were significantly associated with the Reopening Status indicator.

Discussion

A previous study [38] elucidated that health, as a multidimensional issue, should be approached from various perspectives. According to this study [38], the inability of indices like SPAR to accurately predict outcomes of health measures is attributed to the fact that SPAR is an index assessed solely from a health perspective, employing simplistic metrics such as the number of cases or mortality rate. The resulting predictive inaccuracy is also suggested to stem from SPAR’s failure to incorporate indicators regarding the effectiveness of governance in implementing systems within a country. Despite its inaccuracy in predicting health measure outcomes, the study posits that SPAR serves as an instrument capable of identifying gaps, thereby aiding countries in enhancing pandemic preparedness efforts [38].

Our study found that SPAR scores correlate with COVID Resilience Scores (CRS), suggesting that higher SPAR scores correspond to greater resilience. However, among the three CRS indicators, SPAR is significantly associated only with Quality of Life (QoL) and shows no correlation with COVID Status or Reopening Status. This indicates that while SPAR may reflect certain aspects of societal well-being, it does not predict a country’s outbreak severity or policy decisions on reopening.

Although resilience encompasses more than just QoL, our findings suggest that SPAR aligns primarily with this aspect rather than with COVID-19 prevention outcomes. This supports our initial argument that SPAR is not an effective predictor of infectious disease control. A possible reason for this limitation is that SPAR does not adequately account for the social dimensions that are fundamental to resilience.

The lack of correlation with COVID Status may stem from the fact that SPAR measures preparedness capacity rather than real-time outbreak conditions, which are influenced by external factors such as virus mutations and public behavior. Similarly, reopening decisions involve complex political and economic considerations beyond pandemic preparedness. To improve its relevance in future assessments, additional indicators reflecting governance, public trust, and societal adaptability should be incorporated.

Consistent with earlier studies [38, 39], the results suggest that if the outcome of pandemic control is considered solely from the perspective of physical health, SPAR cannot accurately predict the health outcome. However, when viewed from a social perspective [39], SPAR accurately predict the health outcome of pandemic control through the measurement of the community’s quality of life.

Referring to the profound impact of the COVID-19 pandemic on the quality of life of communities [40], the outcome of pandemic control cannot be adequately measured solely by the number of cases and deaths [38]. This aligns with the WHO’s concept of health, which emphasizes assessing health from physical, mental, and social perspectives [41]. Additionally, measuring disease control outcomes from a social perspective aligns with the “balancing dynamic” concept advocated by the International Health Regulations (IHR) of 2005. This concept [42] posits that disease control should continue to consider the community’s quality of life in a balanced manner.

Our study’s findings, indicating that SPAR scores are correlated with CRS, especially the Quality of Life indicator, suggest that a country’s resilience depends on the quality of life of its population. The better the quality of life, the higher the resilience of countries. In the context of a pandemic, it is mentioned that a resilient country effectively controls the pandemic while maintaining economic growth [15, 16]. Thus, in our view, a country with good resilience is one that can implement the “balancing dynamic” concept in controlling the COVID-19 pandemic.

Balance does not always mean equal [43]. Based on theory of balance in health [44], balance is an approach about how to trade off priorities, make choices and achieve harmony across different cultures and historical periods to achieve health. Therefore, it’s understood that how a country applies this balancing dynamic concept depends on its respective situation. Out of the 80 countries we analyzed, only 8 countries have the same score for each CRS measurement indicator, while the other 72 countries have different scores for each CRS indicator.

This pattern persists even among countries with similar characteristics. For example, the highest CRS score obtained by a country in this study is 10, and only 5 countries achieved this score. These five countries have similar profiles, all having high SPAR scores, with four being High-Income Countries (HICs) and only one being a High-Middle-Income Country (HMIC). Although their CRS scores and characteristics may be considered similar, the scores for each indicator in each country are not the same.

In addition to the SPAR, the Government Effectiveness (GE) score is also a contributing factor to the CRS of countries. A higher SPAR score signifies a higher level of country resilience. Previous studies [45, 46] have emphasized the significance of governance in infectious disease control. The quality of GE is related to how effectively and efficiently countries allocate their resources to disease control efforts [6]. Given the extensive scale and prolonged duration of the COVID-19 pandemic [47, 48], the importance of governance effectiveness in pandemic control has become evident. The United Nations (UN) has emphasized the notion that “No one is truly safe until everyone is safe” [49].

Out of the 196 countries that reported their SPAR capacity, this study analyzed only 80 countries, with a majority falling into the category of good governance [34]. This suggests that there are still numerous countries with limited resources and inadequate governance that fail to recognize the impact of pandemic threats and struggle to control them, thereby presenting a concealed danger and hindrance to global pandemic control efforts. This factor is also suspected to contribute to the prolonged duration of the COVID-19 pandemic.

Given that the balance between policies varies across countries [44] and that good governance is crucial for building resilience during health crises, future SPAR assessments should incorporate social elements to help countries strengthen their preparedness for pandemics and other emergencies. One way to achieve this is by encouraging pre-pandemic social discussions that address the prioritization of health measures alongside other societal needs. These discussions, conducted before a crisis occurs, can help establish a more balanced approach to public health policies, ensuring that government responses align with societal values and expectations.

Social discussion refers to the exchange of ideas among individuals or groups and plays a pivotal role in shaping public opinion, fostering cooperation, and promoting understanding within a society. In the context of pandemic preparedness, structured public engagement can help governments navigate the trade-offs between different policy choices—such as balancing disease control measures with economic stability and personal freedoms. Given that pandemic control policies inherently involve competing values [44], it is essential for governments to proactively engage the public in these conversations rather than making unilateral decisions. Policies shaped through broad societal consensus are more likely to be accepted and effectively implemented, ultimately enhancing national resilience. Such decisions should reflect broader societal consensus rather than being solely determined by experts.

Our study suggests that the inaccuracy of SPAR may stem from its limited consideration of social resilience. While Article 2 of the IHR emphasizes balancing health security with economic and social stability, SPAR indicators primarily assess technical and institutional capacities. To address this gap, we propose a new SPAR indicator under Capacity 10 (Risk Communication and Community Engagement), titled “Social Discussion for Policy Balance,” which would assess whether countries proactively engage the public in priority-setting for pandemic control.

This recommendation is particularly relevant to ongoing IHR amendments, which largely focus on verifying national reports rather than expanding resilience assessment criteria. By incorporating pre-pandemic social value communication into the SPAR framework, countries could systematically strengthen public trust and social cohesion—critical components of effective pandemic response.

By addressing this gap, the revised IHR could provide a more comprehensive approach to pandemic preparedness, ensuring that future health security efforts consider not only epidemiological and healthcare capacities but also the broader societal impacts of public health interventions.

Conclusion

The study’s findings indicate that the inaccuracy of SPAR scores is attributable to a narrow focus on disease-related measurements. By solely considering these aspects, the broader social perspective is overlooked. However, when analyzed from a social standpoint, it becomes apparent that the SPAR scores of countries effectively reflect their resilience through the quality of life experienced by their populations. Consequently, the study concludes that the limited incorporation of social aspects within the SPAR framework contributes to the inaccurate prediction of COVID-19 control outcomes.

Since the implementation of the IHR (2005), no outbreak or pandemic—including H1N1 (2009), Ebola (2014–2016), and Zika (2015–2016)—has matched the scale or duration of COVID-19, which lasted for several years. The COVID-19 pandemic was declared in March 2020 and officially ended in May 2023. As a result, SPAR scores have been assessed from a relatively narrow perspective during the pandemic.

Furthermore, the mental and social health impacts of diseases often emerge more gradually than their physical health effects. Given the ongoing discussions on revising the IHR, it is essential to incorporate social aspects into the IHR indicators, particularly those reported through the SPAR system. Enhancing SPAR with social resilience measures would improve its ability to assess countries’ capacities to manage major public health threats and better predict pandemic control outcomes.

Limitations

This study is subject to several limitations, which are as follows. First, the analysis encompassed a limited set of 80 countries owing to a lack of complete data, potentially diminishing the overall significance of this ecological study. Additionally, it is important to acknowledge that variations in the exact timing of countries’ vaccine coverage reporting may introduce potential challenges to data quality. Additionally, measurement errors in COVID-19 status metrics due to underreporting or under ascertainment [50] and other possible unmeasured confounding factors may result in the impact of COVID-19 being underestimated.

Regarding the components used in calculating the CRS, “excessive deaths” was not used in this study due to the concern of limited data and the challenge of its technical barriers [51]. Instead, we used the 3-month case fatality rate (CFR) as one of its components for measuring COVID Status with the further intention of aligning our approach with the Bloomberg version. However, due to its significance, as revealed by the literature [51, 52], future studies may benefit from considering excessive deaths when measuring the outcome of disease control.

In this study, we acknowledge the importance of considering the reliability and validity of the data utilized. It is noteworthy that a previous study [53] has underscored certain limitations and expressed concerns regarding the data quality derived from open-source databases. Despite these acknowledged limitations, we maintained our decision to use the data. This choice is grounded in the widespread use of these resources globally and their frequent utilization as a reference for describing the situations in various countries.

Then, limitation of this study may also come from the process of transforming numeric data into binary variables. Despite the purpose, this conversion comes with certain trade-offs. One notable drawback is the potential loss of granularity and nuanced information present in the original numeric data. The binary classification may lead to a simplification of intricate patterns and variations within the dataset as well as introducing the possibility of information loss or data misrepresentation. Therefore, it is crucial to be aware when interpreting the results, as the binary representation might not capture the full spectrum of the underlying numeric values.

Data availability

No datasets were generated or analysed during the current study.

References

  1. World Health Organization. International health regulations. 2018 [cited 2022 May 30]. Available from: https://www.who.int/health-topics/international-health-regulations

  2. World Health Organization. International Health Regulations. (2005) Third Edition. 2016 [cited 2022 Dec 23]. Available from: https://www.who.int/publications-detail-redirect/9789241580496

  3. Katz R, Dowell SF. Revising the international health regulations: call for a 2017 review conference. Lancet Global Health. 2015;3(7):e352–3.

    Article  PubMed  Google Scholar 

  4. Tsai FJ, Tipayamongkholgul M. Are countries’ self-reported assessments of their capacity for infectious disease control reliable? Associations among countries’ self-reported international health regulation 2005 capacity assessments and infectious disease control outcomes. BMC Public Health. 2020;20(1):282.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Semenza JC, Sewe MO, Lindgren E, Brusin S, Aaslav KK, Mollet T, et al. Systemic resilience to cross-border infectious disease threat events in Europe. Transbound Emerg Dis. 2019;66(5):1855–63.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Satria FB, Tsai FJ, Turbat B, Scientific R. 2022 [cited 2022 Dec 28]. Analyzing self-evaluation capacity scores related to infectious disease control in International Health Regulations during the first year of COVID-19 pandemic. Available from: https://www.nature.com/articles/s41598-022-19361-8

  7. Kandel N, Chungong S, Omaar A, Xing J. Health security capacities in the context of COVID-19 outbreak: an analysis of international health regulations annual report data from 182 countries. Lancet. 2020;395(10229):1047–53.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. World Health Organization. WHO Coronavirus (COVID-19) Dashboard. 2021 [cited 2021 Dec 6]. Available from: https://covid19.who.int

  9. Meyer D, Bishai D, Ravi SJ, Rashid H, Mahmood SS, Toner E, et al. A checklist to improve health system resilience to infectious disease outbreaks and natural hazards. BMJ Global Health. 2020;5(8):e002429.

    Article  PubMed  PubMed Central  Google Scholar 

  10. UNDESA, Everyone Included. Social Impact of COVID-19| DISD. 2022 [cited 2022 May 14]. Available from: https://www.un.org/development/desa/dspd/everyone-included-covid-19.html

  11. World Health Organization. WHO Coronavirus (COVID-19) Dashboard. 2022 [cited 2022 Dec 28]. Available from: https://covid19.who.int

  12. Our World in Data. COVID-19 Pandemic. 2025 [cited 2025 Feb 16]. Available from: https://ourworldindata.org/coronavirus

  13. UNDRR. Review of COVID-19 Disaster Risk Governance in Asia-Pacific. Towards Multi-Hazard and Multi-Sectoral Disaster Risk Reduction. 2020 [cited 2022 May 21]. Available from: https://www.undrr.org/publication/review-covid-19-disaster-risk-governance-asia-pacific-towards-multi-hazard-and-multi

  14. The World Bank. World Bank. 2020 [cited 2022 Mar 30]. COVID-19 to Plunge Global Economy into Worst Recession since World War II. Available from: https://www.worldbank.org/en/news/press-release/2020/06/08/covid-19-to-plunge-global-economy-into-worst-recession-since-world-war-ii

  15. Seddighi H. COVID-19 as a natural disaster: focusing on exposure and vulnerability for response. Disaster Med Public Health Preparedness. 2020;14(4):e42–3. https://www.cambridge.org/core/journals/disaster-medicine-and-public-health-preparedness/article/covid19-as-a-natural-disaster-focusing-on-exposure-and-vulnerability-for-response/5C2290D183E673E9B7E0B0CD3F6505AB?utm_campaign=shareaholic%26utm_medium=copy_link%26utm_source=bookmark

  16. Center for Disaster Philanthropy. Resilience. 2022 [cited 2022 May 20]. Available from: https://disasterphilanthropy.org/resources/resilience/

  17. Bloomberg, Methodology. Inside Bloomberg’s Covid Resilience Ranking. 2020 [cited 2022 Jan 26]. Available from: https://www.bloomberg.com/news/articles/2020-11-24/inside-bloomberg-s-covid-resilience-ranking

  18. Hanvoravongchai P, Adisasmito W, Chau PN, Conseil A, de Sa J, Krumkamp R, et al. Pandemic influenza preparedness and health systems challenges in Asia: results from rapid analyses in 6 Asian countries. BMC Public Health. 2010;10(1):322.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Ritchie H, Mathieu E, Lodes-Guirao L, Apple C, Giattino C, Ortiz-Ospina E et al. Share of people who received at least one dose of COVID-19 vaccine. 2022 [cited 2022 Apr 11]. Available from: https://ourworldindata.org/covid-vaccinations

  20. Ritchie H, Mathieu E, Rodés-Guirao L, Appel C, Giattino C, Ortiz-Ospina E et al. COVID-19: Stringency Index. Our World in Data. 2020 Mar 5 [cited 2022 Jan 21]; Available from: https://ourworldindata.org/covid-stringency-index

  21. Ritchie H. Our World in Data. 2020 [cited 2022 Mar 2]. Google Mobility Trends: How has the pandemic changed the movement of people around the world? Available from: https://ourworldindata.org/covid-mobility-trends

  22. Humanitarian Data Exchange (HDX), Office for the Coordination of Humanitarian Affairs (OCHA). COVID-19 Global Travel Restrictions and Airline Information. 2022 [cited 2022 Mar 23]. Available from: https://data.humdata.org/dataset/covid-19-global-travel-restrictions-and-airline-information

  23. Our World in Data. Our World in Data. 2022 [cited 2022 Jan 20]. COVID-19 Data Explorer. Available from: https://ourworldindata.org/coronavirus-data-explorer

  24. World Health Organization. WHOQOL - Measuring Quality of Life|. 2012 [cited 2022 Apr 13]. Available from: https://www.who.int/tools/whoqol

  25. World Population Review, World Happiness Report. 2021. Happiest Countries in the World 2021. 2022 [cited 2022 Jan 12]. Available from: https://worldpopulationreview.com/country-rankings/happiest-countries-in-the-world

  26. Vision of Humanity. Vision of Humanity. 2022 [cited 2022 Jan 12]. Global Peace Index Map, The Most & Least Peaceful Countries. Available from: https://www.visionofhumanity.org/maps/

  27. The World Bank. Annual GDP Growth. 2022 [cited 2022 Jan 26]. Available from: https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG

  28. United Nations Development Program (UNDP). Human Development Reports. 2022 [cited 2022 Jan 12]. Available from: http://hdr.undp.org/en/content/download-data

  29. Roser M. Human Development Index (HDI). Our World in Data. 2014 Jul 25 [cited 2022 Apr 11]; Available from: https://ourworldindata.org/human-development-index

  30. World Health Organization. e-SPAR Public. 2019 [cited 2022 Jan 12]. Available from: https://extranet.who.int/e-spar

  31. Freedom House. Freedom House. 2021 [cited 2021 Aug 18]. FAQ - Freedom in the World. Available from: https://freedomhouse.org/reports/freedom-world/faq-freedom-world

  32. Freedom House. Freedom House. 2021 [cited 2021 Aug 18]. Freedom in the World Research Methodology. Available from: https://freedomhouse.org/reports/freedom-world/freedom-world-research-methodology

  33. Freedom House. Freedom House. 2021 [cited 2021 Aug 18]. Freedom in the World. Available from: https://freedomhouse.org/report/freedom-world

  34. The World Bank. WGI 2020. 2020 [cited 2021 Feb 5]. Available from: http://info.worldbank.org/governance/wgi/

  35. Tsai FJ, Turbat B. Is countries’ transparency associated with gaps between countries’ self and external evaluations for IHR core capacity? Globalization Health. 2020;16(1):10.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Hamouda O. The role of the federal government in the prevention and control of infectious diseases. Organizational structures, responsibilities and duties–an overview]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2005;48(9):963–70.

    Article  CAS  PubMed  Google Scholar 

  37. Kenis P, Schol LGC, Kraaij-Dirkzwager MM, Timen A. Appropriate governance responses to infectious disease threats: developing working hypotheses. Risk Hazards Crisis Public Policy. 2019;10(3):275–93.

    Article  Google Scholar 

  38. Stoto MA, Nelson CD, Kraemer JD. Does it matter that standard preparedness indices did not predict COVID-19 outcomes? Globalization and Health. 2023 [cited 2024 Jan 8];19(1). Available from: https://link.springer.com/epdf/https://doi.org/10.1186/s12992-023-00973-2

  39. Fukuda-Parr S. When indicators fail: SPAR, the invisible measure of pandemic preparedness. Policy Soc. 2022;41(4):528–40.

    Article  Google Scholar 

  40. Hansel TC, Saltzman LY, Melton PA, Clark TL, Bordnick PS. COVID-19 behavioral health and quality of life. Sci Rep. 2022;12(1):961.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. World Health Organization. Constitution of the World Health Organization. 2022 [cited 2022 May 14]. Available from: https://www.who.int/about/governance/constitution

  42. Gostin LO, Katz R. The international health regulations: the governing framework for global health security. Milbank Q. 2016;94(2):264–313.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Oxford Advanced Learner’s Dictionary. Balance. 2024 [cited 2024 Jan 11]. Available from: https://www.oxfordlearnersdictionaries.com/definition/english/balance_1?q=balance

  44. Lipworth WL, Hooker C, Carter SM. Balance, balancing, and health. Qual Health Res. 2011;21(5):714–25.

    Article  PubMed  Google Scholar 

  45. Menon-Johansson AS. Good governance and good health: the role of societal structures in the human immunodeficiency virus pandemic. BMC Int Health Hum Rights. 2005;5(1):4.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Blackwood JC, Malakhov MM, Duan J, Pellett JJ, Phadke IS, Lenhart S, et al. Governance structure affects transboundary disease management under alternative objectives. BMC Public Health. 2021;21(1):1782.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Johns Hopkins Medicine. Coronavirus Second Wave, Third Wave and Beyond: What Causes a COVID Surge. 2022 [cited 2023 Feb 22]. Available from: https://www.hopkinsmedicine.org/health/conditions-and-diseases/coronavirus/first-and-second-waves-of-coronavirus

  48. Agarwala P, Bhargava A, Gahwai DK, Negi SS, Shukla P, Dayama S. Epidemiological Characteristics of the COVID-19 Pandemic During the First and Second Waves in Chhattisgarh, Central India: A Comparative Analysis. Cureus. 2022 Apr [cited 2023 Feb 22];14(4). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9106595/

  49. United Nations. United Nations. United Nations. 2021 [cited 2023 Jul 4]. No one is safe, until everyone is. Available from: https://www.un.org/en/desa/%E2%80%9Cno-one-safe-until-everyone-%E2%80%9D

  50. Markovic S, Salom I, Rodic A, Djordjevic M. Analyzing the GHSI puzzle of whether highly developed countries fared worse in COVID-19| Scientific Reports. 2022 [cited 2023 Oct 8]. Available from: https://www.nature.com/articles/s41598-022-22578-2

  51. Sagan A, Thomas S, Kennedy E, Webb E, McKee M. Assessing resilience of a health system is difficult but necessary to prepare for the next crisis| The BMJ. 2023 [cited 2023 Sep 11]. Available from: https://www.bmj.com/content/382/bmj-2022-073721

  52. Ledesma JR, Isaac CR, Dowell SF, Blazes DL, Essix GV, Budeski K, et al. Evaluation of the global health security index as a predictor of COVID-19 excess mortality standardised for under-reporting and age structure. BMJ Global Health. 2023;8(7):e012203.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Stoto MA, Woolverton A, Kraemer J, Barlow P, Clarke M. COVID-19 data are messy: analytic methods for rigorous impact analyses with imperfect data. Globalization and Health. 2022 [cited 2024 Jan 8];18(1). Available from: https://link.springer.com/epdf/https://doi.org/10.1186/s12992-021-00795-0

Download references

Acknowledgements

This research has been presented at two conferences. The first presentation took place at the Taiwan Public Health Joint Annual Conference in Taipei, Taiwan, in October 2022. The second presentation was held at the Consortium Universities of Global Health (CUGH) 2023 conference in Washington, DC, in April 2023.

Funding

National Science and Technology Council, Republic of China (113-2410-H-038-009-MY2).

Author information

Authors and Affiliations

  1. Philosophy Doctor in Medicine Program, Faculty of Medicine, Universitas Sumatera Utara, Medan, Indonesia

    Fauzi Budi Satria

  2. Department of Community Medicine, Faculty of Medicine, Universitas Sumatera Utara, Medan, Indonesia

    Fauzi Budi Satria

  3. Regional Collaborating Centre, Universitas Sumatera Utara– Singhealth DukeNUS Global Health Institute (USU– SDGHI), Medan, Indonesia

    Fauzi Budi Satria

  4. Ph.D. and Master program in Global Health and Health Security, College of Public Health, Taipei Medical University, 250 Wu-Xing Street, Taipei, 110, Taiwan

    Feng-Jen Tsai

Authors
  1. Fauzi Budi Satria
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Feng-Jen Tsai
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Both FBS and FJT made equal contributions to the development of the protocol, conducted the literature searches, extracted and analyzed the data, and interpreted the results. FBS took the lead in drafting the manuscript, incorporating suggestions from FJT. Both authors carefully reviewed and approved the final version of the manuscript. FJT assumes responsibility as the corresponding author of the paper.

Corresponding author

Correspondence to Feng-Jen Tsai.

Ethics declarations

Ethics approval and consent to participate

Not Applicable. This study did not involve human subjects, as it relied solely on secondary data reported by national authorities and was therefore exempt from ethical review. The Belmont Report acknowledges that research using publicly available data may not require ethical review, as it does not involve interactions with individuals or the collection of identifiable private information.

Consent to participate

Not applicable as no human subject was being used in this study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Patient and public involvement

Patients and/or the public were not involved in this research.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Satria, F.B., Tsai, FJ. Why do International Health Regulations self-assessment capacities (SPAR) scores not predict COVID-19 control outcomes? – analysis of the relationship between SPAR scores and COVID-19 resilience scores in 2021. Global Health 21, 19 (2025). https://doi.org/10.1186/s12992-025-01111-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12992-025-01111-w

Keywords

Globalization and Health

ISSN: 1744-8603