Stakeholder perceptions and public health system performance evaluation: Evidence from Greece during the COVID-19 pandemic

Introduction

One of the most immediate and sustained effects of the COVID-19 pandemic has been the challenging and testing of public health systems across the globe. In every domain, changes in environmental parameters force systems to adjust, adapt, extend or, in some cases, to destabilize and break down. The COVID-19 threat, defined by intrinsic ambiguity and resilience, has engaged public health systems in a prolonged scrutinization of their functions, capacity and resources. In theory, this process can yield invaluable insights for future policy design and mitigate future adversity. However, as comprehensively rational policymaking has long been proven an unattainable aspiration, scholars and practitioners are forced to admit that paradigms are dynamically shaped by the agents who design and execute them. Very often, innovative and ambitious legislative frames are a far cry from policymaking realities plagued with institutional and operational deficiencies. As a result, we decide to move past assessments of the de jure status quo and examine the de facto modus operandi through the eyes of its protagonists.

In specific, we focus on the case of Greece and set out to answer whether the performance of the country's Public Health System manages to fulfill its intended aims and manage outstanding challenges. We define a Public Health System as a set of interacting processes and instruments functioning for the prevention, promotion and protection of the population's health. We zone in on the latent dimension of “stakeholder perceptions” and report insights obtained directly by stakeholders in the Greek public health system collected between 15.07.2020 and 13.12.2020. Greece, since 2003, boasts a modern public health system design, aligned with contemporary public health thought and international trends. Facing COVID-19, the Greek system came face-to-face with a public health crisis of unprecedented magnitude, bringing its functioning to the forefront after long and calling for an assessment of its performance. Guiding our analysis is a refined version of the public health system performance evaluation framework introduced by Handler and Turnock (2001). We propose a re-operationalization of the framework's components to orient the assessment toward policy conclusions rather than universal systemic metrics. We incorporate the effect of the COVID-19 pandemic both in an indirect fashion–through the timing of our survey–and a direct one, through explicit questions. As such, we seek to contribute new perspectives to assessing public health systems' performance, fill in literature gaps on stakeholder insights (Montante, 2022) and bring in unique evaluative conclusions for the Greek case, at a time when the country's public health system has been facing its greatest test to date. First, we rationalize our approach, then we report findings from our data and finally we present conclusions, policy insights and future research directions.

Evaluating public health systems

The archetype of the modern public health system is a recent development in history. The founding principles which currently underpin public health services provision in western democracies were not introduced until the 1970's. For most of the 21^st century, improving public health was predominantly pursued through developing, sustaining and improving biomedical health care systems and technologies. However, after the publication of a series of policy papers across the globe between 1974 and 1979,¹ it became understood that the established systemic paradigm had been both deficient and misoriented. Given the complexity and dynamism of the human existence, it is imperative “to look beyond the traditional health care (sick care) system if we wish to improve the health of the public” (Hancock, 1985).

Embodying this principle, the “New Public Health” movement has since garnered international acceptance. Following the 1^st World Health Organization Conference on Health Promotion in Ottawa (1986), the provision and sustaining of peace, shelter, education, food, income, stable eco-systems, resources, social justice and equity have joined therapeutic methods as drivers behind “Health for All.” Recognizing the impact of the “social determinants” of health renders policymaking multidimensional, with channels toward public health outputs ranging from “social and economic policies” and “institutions” to “pathopsychological pathways” (Smedley and Syme, 2000). Ultimately, to best utilize resources and increase the probability of favorable outcomes, it becomes imperative for public health systems to manage socioeconomic, environmental and cultural conditions, social and community networks, individual lifestyle factors and demographic parameters (Dahlgren and Whitehead, 1991). “In public health, in contrast to clinical practice, the whole society is your patient” (Tom Frieden, US CDC Director, 2009-2017).

Evaluating public health systems today demands looking past the mere reading of unidimensional epidemiological indicators and instead calls for examining the co-dependencies between the system's various aims, means and outcomes. As early as in 1980, Donabedian laid the blueprint for such an approach, viewing quality of care as the product of an interplay between three equally crucial parts: systemic structures (staff, equipment, guidelines, supplies), processes (interaction, investigation, examination/treatment, counseling) and outcomes (morbidity, mortality, satisfaction). Donabedian's contribution, although innovative, presented a rather narrow operationalization of processes and lacked any form of dynamic ranking between the model's components. Two decades later, Handler and Turnock (2001) presented a new framework to assess public health system performance, building on Donabedian's insights. They redefined processes to comprise of the “the 10 essential public health services,”² identified groups of resources making up structural capacity, revisited appropriate outcome measures and added the system's mission and the macro context as essential components. The former describes the goals of a public health system and their conceptual operationalization while the latter captures the supra-system social, political, and economic forces which affect its functioning. Changes in the macro context can shape all aspects of the public health system while its mission guides the development of processes and the building up of structural capacity. Last, a contribution by Shi et al. (2009) proposes a slightly amended iteration, with the macro context assumed to also impact outcomes directly; not just through the system. This way, the framework lies closer to the true nature of the social determinants of health as policymaking in any sector may determine public health outcomes.

An adapted visualization of the framework by Handler and Turnock (2001) including the amendments by Shi et al. (2009) is presented below (Figure 1).

Figure 1. The public health system performance evaluation framework (Handler and Turnock, 2001; Shi et al., 2009).

The described rationale successfully captures the general structure of modern public health systems while allowing for points of divergence across cases to be accommodated. However, the framework still demands some refinement to transition from an “ideal-type” mapping of input and output flows to an analytical guide for understanding and explaining public health system functioning. First, while concentrating on “effectiveness, efficiency and equity” as the key outcome variables allows the extraction of generalizable and easily comparable data, it poses content validity threats by reducing public health assessments to universal systemic performance metrics. Second, evaluating the presence or the degree of completion of “essential processes” across systems with potentially diverging missions reduces the policy usefulness of insights. Undoubtedly, converging toward protecting and promoting all recognized elements which foster good population health should be the ultimate goal of every public health system. However, in understanding success and failures, assessments should definitively answer the following fundamental question: Are the system's outcomes, pursued through the system's processes and structures within the given macro context, achieving the system's intended mission? Only then can research pinpoint deficiencies and deduce whether adjustments should be oriented toward better policy goals, better-functioning policy means or both.

As a result, when moving past the technical elements of performance and toward analyses of policy fit, a more focused operationalization of the framework's components emerges as an essential challenge. Insightful conclusions can be produced through research designs which revisit both the source and content of data. Large-N studies which compare structural capacity and (relevant) policymaking input channels between countries can drive policy learning based on relative institutional adequacy. In the single case study level, elite surveying with policymaking participants can uncover areas of underperformance despite seemingly well-designed processes and evaluate whether those who drive and execute policy embrace the established paradigm. In times of crisis like the COVID-19 pandemic, with divergent national responses and accelerated policy reform timeframes, the latter approach emerges most interesting and fruitful.

A revised framework in practice: The case of Greece

We propose a refined version of the public policy performance evaluation framework and apply it to the case of Greece. Greece, since 2003, boasts a progressive legal framework for public health, founded on the holistic perspective to health. Importantly, this was the outcome of a disruptive rather than incremental evolution. Throughout the 1980's and 1990's Greece remained stuck on a medicine-oriented paradigm centered around a public system of secondary services, the Greek National Health System (GNHS). Until 2003, public health policymaking lacked any dedicated bill, much less a dedicated system. Eventually, the marquee legislation, bill 3172/2003 was introduced, was complemented with bill 3370/2005 2 years later, and stood unmodified until the start of the COVID-19 pandemic. A further revision was passed amidst the pandemic's first wave in March 2020 (bill 4675/2020), focusing mostly on updating organizational matters and the field's institutional mapping. Overall, public health in Greece, for the past two decades, has been defined as “the sum of the organized actions by the state and the society which aim at the promotion of health, the improvement of the quality of life and the increase of life expectancy for the entirety of the Greek population” (bill 3172/2003).

Attributing a whole-of-society approach to the pursuit of public health outcomes has underpinned a systemic design with intersectoral instruments and highly diverse functions and processes. With legislative revisions having followed the same policy direction, the de jure infrastructure of the Greek public health system has appeared harmonized with modern public health policy thought. We seek to dive deeper and explore the de facto reality of the system's performance using the insight of agents who steer its operational and executive functioning.

We interpret survey data collected from 261 public health policy stakeholders in Greece between 15.07.2020 and 13.12.2020. In line with the latest scholarship on elite surveying, we employed a mixture of purposive and quota sampling during data collection to maximize response rates and ensure representative participation from an intrinsically small-N sampling frame (Acharya et al., 2013; Etikan et al., 2016). We decided on sampling criteria after a thorough mapping of stakeholders in the Greek public health policy community, informed by the legal framework (bills 3172/2003, 3370/2005, 4675/2020) and two Action Plans for Public Health published by the Ministry of Health in 2008 and 2019. Public health bills describe the participating institutions of the Greek public health system and action plans present in detail the public and non-public stakeholders with input and involvement in public health policy. As such, we were able to minimize proximity bias—i.e., overreliance on easily accessible stakeholders–as well as to proportionally incorporate interviewees from small sampling frames—e.g., the health policy media.

Taking the stakeholder distribution into account, we set as the primary criterion the equal and analogous representation of 85% of our sample from stakeholders in the Public Sector (participants in political and administrative positions in the state mechanism), the Private Sector (private doctors, pharmacists and members of the industry—e.g., pharmaceuticals—) and Research Centers/Universities and the equal and analogous representation of the remaining 15% of our sample from participants in NGOs/Patients Associations and the Media. Moreover, seeking to include both veteran and early-career stakeholders, we set as a secondary criterion the equal and analogous representation of participants in the over 60, under 30 and 31–40 age groups for 30% of the sample and the equal and analogous representation of participants in the 41–50 and 51–60 age groups for the remaining 70% of the sample. Given the small sample and the nature of the study, the analysis focuses predominantly on aggregate rather than subgroup-correlational results. As a result, the construction of a representative sample is of utmost importance. Ultimately, our participants fitted the intended profile (Figure 2), while the response rate was 52% (261/503).

Figure 2. Sample distribution (occupation and age).

In collecting and processing the data, we design and operate a modified version of the public health system performance evaluation framework. We identify that the work of Donabedian (1980), Handler and Turnock (2001) and Shi et al. (2009) lies close to the operational logic of “black box” frameworks, popularized by Easton (1953). Recognized as one of the founders of modern public policy thought, Easton proposed that how political systems turn inputs–demands–to outputs–decisions or policies–depends on a contextually unique set of processes–a black box–which researchers should uncover and understand. Crucially, outputs, through internal feedback, and the macro environment, through external feedback, influence supports, i.e., agent preferences and interests, which are essential to both policy adoption and implementation. Echoing this logic, we include supports to our analysis through our selection of data and capture the effect of the COVID-19 pandemic in two ways. First, opting to collect stakeholder's insights on the performance of the Greek public health system during the pandemic's first year, we incorporate the inherent “internal feedback” dimension and second, recognizing the pandemic as a macro-context evolution, we capture the “external feedback” dimension through explicit dedicated questions. We thus propose both a new operationalization of variables and new evaluation metrics.

Last, we set out two hypotheses to test during our study, inspired by the literature and aligned with the analytical capacity of the framework. Taking into account that the current Greek Public Health policy paradigm emerged with little incremental building-up and that policy change was heavily intertwined with a broader agenda for governance modernization (Mavrikou, 2021), we expect our evaluation to illuminate that the underpinning policy principles have not been sufficiently internalized by the participating stakeholders. Moreover, given the traditionally subpar production and intake of policy evaluation insights in Greece (Lionis, 2009; Spanou, 2020), we expect the COVID-19 pandemic to not only highlight new challenges for the Greek Public Health System but also to surface long-standing, unaddressed problems. We summarize our hypotheses below and present our analysis in the following section.

H1: The holistic paradigm for health, provisioned to underpin the functioning of the Greek Public Health System, has not been fully embraced by the system's participating agents.

H2: The COVID-19 pandemic is likely to not only bring new challenges to light, but surface long-standing unaddressed problems of the Greek Public Health System in the evaluations of stakeholders.

Analysis

The macro context

The impact of the macro context on public health systems is threefold. First, from an agenda-setting and policy formulation standpoint, it determines the systemic design; the selection of aims, processes and relevant resources. Second, from an operational and evaluative standpoint, it defines inputs external to the system that are perceived as important to its functioning and the delivering of outcomes. Third, when large-scale macro context developments test the public health system, the emergence of new challenges induces a process of testing which can generate pressure for adjustments or changes. Given the scope of this study, we are interested in the latter two. Before explicitly evaluating the system's behavior in the face of the COVID-19 pandemic, we explore stakeholder perceptions regarding the wider set of factors which influence the population's quality of health (Figure 3).

Figure 3. Impact of determinants on the population's quality of health–mean stakeholder evaluations.

We asked respondents to evaluate in a 10-point scale, from 0, no impact, to 10, maximum impact, the degree to which a series of determinants influence the population's health. We relied on accepted classifications of health determinants in the literature to inform our options (Marmot and Wilkinson, 2005; Braveman and Gottlieb, 2014). In their response, Greek public health policy stakeholders view access to public health services as the most impactful determinant, with a mean evaluation of 8.79. The uncontestably endosystemic option ranks first, denoting that the responsibility of the dedicated services in place is both recognized and accepted among the policy community. Nevertheless, seven other options receive mean evaluations over seven 7 of 10. They are income (7.81), the natural environment (7.70), education (7.69), occupation (7.23), the family environment (7.22) and the work environment (7.18). Prevailing beliefs in Greece acknowledge macro-contextual influence across domains: from employment and income to environmental and social conditions, with mean scores between options being similar. Health outcomes are predominantly linked with the presence and function of the system itself, but developments in other core policymaking areas are understood to play an essential role; this is important insight given the multifaceted nature of the COVID-19 crisis.

The mission of the Greek public health system

The mission and purpose of the Greek public health system is synthesized through the relevant provisions of the three dedicated public health bills introduced in 2003, 2005, and 2020. In specific, we identify nine core aims which are set to govern the operation of the system. The identified systemic aims describe the intended outcomes which the interaction of processes, resources and macro-contextual developments should deliver (Hall, 1993; Handler and Turnock, 2001). First, lying close to the traditional scope of public health policymaking, are goals for the prevention and promotion of health, the management of infectious diseases, the application of scientific advancements in medicine for the protection of public health, the measuring and monitoring of the population's quality of health and the measuring and improving of the effectiveness of health services. Moreover, reflecting the holistic perspective to health, goals for dealing with extraordinary circumstances (natural disasters/epidemics), setting aims for the population's quality of health, assessing socio-economic determinants and containing health inequalities are also included. The identified aims fall in line with theoretical and policy developments of the first two decades of the 21^st century regarding the areas of responsibility for public health systems and even borrow heavily from highly recognized templates (e.g., World Health Organization Regional Office for Europe, 1999; European Observatory on Health Systems Policies. Papanicolas and Smith, 2013). However, insights from the evaluation will determine their degree of fulfillment as well as priorities for readjustment given the extraordinary circumstances of the COVID-19 pandemic.

In addition to the defined aims, the Greek legislative framework provides a conceptualization of public health as a notion and a policy field. In specific, it proposes that public health should be most closely synonymous to evaluating health needs and eliminating health threats and include but, importantly, exceed as a concept the fields of prevention or primary, secondary, post-hospital, pharmaceutical and long-term care. Given that defining the notion of public health sets the ideational foundation of the system's paradigm, it is crucial to explore convergence or divergence among the legislative provisions and the outlook of participating agents when identifying the system's philosophy. To this end, we asked stakeholders to select three options which best describe the nature of public health (Figure 4).

Figure 4. Areas which best describe the nature of public health.

The aggregate responses show conflict in the conceptualization of public health among the de jure and the de facto operational frame of the Greek system. The traditional services-oriented perspective to public health seems to dominate among respondents, with 73.18% choosing prevention as one of the three core dimensions, 56.70% selecting secondary care and 42.53% primary care. The two principles which best describe the nature of public health according to the legislative framework—evaluating health needs and containing and eliminating health threats—are selected by only 36.02 and 34.48% of respondents respectively. Despite modern aims, public health in Greece remains viewed through a care-focused lens, hinting at a misoriented philosophy being espoused by the agents in charge of steering outcomes.

The processes of the Greek public health system

Through synthesizing the relevant legislative provisions, nine core functions are identified in the Greek public health system. In contrast to aims, functions present the grouping of processes which individual instruments or groups (in a subsystem fashion) are mandated to carry out in order to fulfill the system's mission. Similar to how “the 10 essential public health services” were used as umbrella proxies for the processes of the ideal-type public health system in the work of Handler and Turnock (2001), we classify the sets of processes identified in our case study into functions. Hence, while aims speak of intended outcomes, functions speak of defined intertwined processes Eliminating health inequalities, quality-checking food and drink, designing and implementing vaccination programs and managing extraordinary circumstances (natural disasters/epidemics) tie in with the country's traditional infrastructure in secondary care, primary care, prevention, hygiene and crisis management. Moreover, promoting the adoption of healthy behavioral patterns is a function that has been attributed increased importance in the 2020 legislative revision. The remaining four functions—limiting poverty and social exclusion, increasing employment, protecting the natural environment and improving working conditions—are highly connected to aims for assessing and managing the impact of socio-economic determinants to the population's quality of health and involve agencies and institutions well beyond the boundaries of the health sector.

Overall, the processes of the Greek health public health policy system resemble the archetype of “essential public health services” (CDC, 1994) but also show divergence. First, the Greek system appears to only approach a skilled workforce through the resources component of the system, lacking dedicated processes to continuously improve and specialize public health experts. On the other hand, as public health in the Greek legal framework is attributed a level of breadth which captures virtually all aspects of public policymaking, the “use of legal and regulatory actions” or “the creation, championing and implementation of policies, plans and law” serve as foundational principles rather than processes in the system. Instead, functions which arguably provision more diverse and holistic public health policymaking and regard the natural environment, poverty and working conditions are included. While both classifications exude a similar perspective to the operation of the public health system, non-trivial differences render identifying the particularities of processes on a case-specific basis integral for a targeted evaluation.

Nevertheless, while functions appear comprehensive at the legislative level, the participating agents of the system do not appear to acknowledge the mandate for all of them equally. We asked our respondents to select options which make up the functions of public health policymaking in Greece, allowing any number or combination of selections (from one to all of them) (Figure 5).

Figure 5. Functions of public health policymaking.

Strikingly, four options were selected by less than half of respondents, with all of them being the non-traditional functions which extend the health policy sector. Protecting the environment was selected by 39.46%, limiting poverty and social exclusion by 38.70%, improving working conditions by 29.12% and increasing employment by just 8.81% of participants. On the contrary, the design and implementation of vaccination programs appears to uncontestably enjoy the highest levels of acceptance—selected by 85.06%—, followed by eliminating health equalities−71.26%—. Moreover, between 55 and 57% recognize the promotion and adoption of healthy behavioral patterns, the quality-checking of food and drink and the management of extraordinary circumstances (natural disasters/epidemics) as processes under the responsibility of the system.

In line with the insights on the conceptualization of public health as a notion and a policy field, the holistic paradigm which Greece's public health bills reflect seems to not be internalized by large portions of the policy community. The stakeholders of the Greek public health policymaking system maintain a more restricted outlook, oriented toward traditional functions, and focused on the access and quality of care and hygiene services. The absence of mandates for the development of public health specialization likely reinforces this perception. This finding hints at failed policy change in practice, as provisioned processes have either not been established or are not actively driven forward by policymaking agents. Divergence in the identification of systemic functions likely impacts outcomes by generating deficiencies in the pursuit of aims.

The resources of the Greek public health system

Supporting the execution of processes are resources. Resources may be generated at the central, subsystem and organizational level and vary in importance and contribution depending on the function or process they support. Intending to retain the focus of the analysis at the systemic level, we evaluate resources through the insight of Greek public health stakeholders as follows: First, we are interested in the underappreciated component of supervising responsibility. Second, we focus on identified categories of resources according to the provisions of the relevant legal framework. Last, as public health policymaking is institutionally recognized as an intersectoral evidence-based field, we deep-dive into the scientific informational resources which respondents deem necessary for the system's functioning and advancement.

Supervising responsibility touches on the diversity of public health systems across countries. While public health policymaking may reflect similar aims, the organization of systems tends to exhibit variance between states, depending on broader features of the political landscape. For example, Scandinavian countries, like Sweden, attribute increased responsibilities to independent public organizations, the UK relies highly on strong regional instruments and the USA limits direct state involvement and increases individual responsibility (Resnik, 2007; Guldbrandsson and Fossum, 2009; Boyle, 2011). Intrinsically, the holistic paradigm to health demands the presence of leading instruments at both the central and the regional/local level to effectively promote functions such as monitoring and assessing health needs. In the case of Greece, legal provisions attribute high importance to the function of the central government, speak of Regional Health Authorities, delegate primary care to the local level, assign higher levels of responsibility to individual attitudes since the 2020 legislative revision and even recognize the contribution of non-governmental organizations.

Nevertheless, among stakeholders, public health policymaking remains most closely intertwined with the actions of the central government, as argue 87.36% of respondents (Figure 6). Regional administrative instruments are recognized as the second most important entities with responsibility—selected by 31.03%—and are followed by municipalities—selected by 27.58%—. NGO's rank far last among participant choices as they are selected by only 2.68% of respondents. While the ranking of options reflects the institutional organization of the Greek public health system, the almost consensus selection of the central government as a chief responsible entity shows a strong top-down policy frame. Greece has perennially struggled with developing decentralized and independent administrative instruments with non-implementation, politicization and Ministerial dominance being intrinsic features of the country's policymaking in health (Mavrikou, 2021). As highly centralized systems struggle with parallel processing, this top-heavy concentration threatens the proportional development of the system's processes and the handling of multifaceted crises such as the COVID-19 pandemic.

Figure 6. Entities responsible for public health policymaking.

At the aggregate level, the Greek public health system recognizes a series of fiscal, human, managerial and informational resources which should contribute to the system's functioning and the completion of its aims. Among stakeholders of the policy community, fiscal resources are recognized as the most essential, being selected in the relevant question by 74.33% of the sample (Figure 7). Given Greece's recent exit from a 10-year economic depression, long-term underfunding has made fiscal resources particularly scarce (Mavridoglou and Polyzos, 2022). Furthermore, in the distribution of spending, 44% of the 2021 health budget was devoted to impatient care while only 1.4% was used for preventive services, among the lowest between EU countries (OECD/European Observatory on Health Systems Policies, 2021). Scientific justification and research closely follow with 64.37%, highlighting that stakeholders recognize the evidence-based nature of the system's operation which the institutional framework promotes. Management, human capital, and public-private partnerships follow at 46.36%, 45.98% and 38.70% respectively, while quantifiable aims receive 19.16%. Interestingly, both in the relevant legislative provisions and among participant opinions, the high importance attributed to scientific programming is not accompanied with measurable goals; undermining the evaluation and improvement of processes. Finally, together with the low recognition of the responsibility of non-state actors, the involvement of NGOs and patient associations as supporting resources is also appreciated lowly–being selected by only 5.75 and 5.36% of respondents respectively.

Figure 7. Contributing resources in the Greek public health system.

Zoning in on response profiles based on occupation uncovers interesting discrepancies in priorities. On the one hand, the importance of funding is uncontestably recognized. However, scientific justification and research is selected by 80.88% of respondents in universities and research centers compared to 59.21% of public sector employees, human capital is selected by 52.63% of public sector employees compared to ~42% of respondents in universities, research centers and the private sector and public-private partnerships are chosen by 47.44% of private sector employees compared to 26.47% of researchers (Figure 8). While the observed alignment between priorities and occupation is intrinsic, it highlights a clear divergence in preferences between key sets of stakeholders within the system. Ultimately, it appears that all groups demand resources close to their specialization to allow their increased involvement. However, convergence in recognizing the essentiality of such key resources is necessary for the system's operating agents to contribute to generating them.

Figure 8. Contributing resources in the Greek public health system–comparison by respondent occupation.

Last, zoning in on informational resources can give insights on whether the multidimensional functions of the Greek public health system are adequately and proportionally supported (Figure 9). As it emerges, the insights of participants align with the care-centric perspective perforating the evaluation thus far. In a three-option combination, medicine, most closely related to secondary care, epidemiology, most closely associated with prevention, and health economics, most closely connected to care services and pharmaceutical products reimbursement are recognized as the scientific fields which best support public health policymaking according to 85.06, 82.38, and 49.04% of respondents respectively. Statistics, which tie into quantifiable aims and evaluative processes are selected by 37.16% while communicative, psychological, and social sciences (sociology and political science) are all chosen by < 17% of respondents. As resources drive processes to deliver outcomes, there is a highly noticeable ill-alignment between the acknowledged and the provisioned inputs to the system. Care-centric resources support care-centric outcomes and impede the functioning of processes which strive to manage the influence of social determinants on health. The impact of this identified divergence to the system's outcomes is evaluated in the next section.

Figure 9. Contributing scientific resources to the functioning of the Greek public health system.

The outcomes of the Greek public health system

As outlined in the beginning of the analysis, we evaluate whether the mission of the Greek public health system is served through its outcomes using the insight of participating stakeholders. To this end, we explore the degree of completion of the system's provisioned aims as well as capture the ideational dimension, i.e., whether the holistic perspective to health underpinning the established paradigm is pursued through the system's current operation.

Participants of the Greek public health policy community paint a stark picture regarding the system's performance (Figure 10). In a scale from 1–the aim of the system is not fulfilled at all–to 10–the aim of the system is fulfilled completely—, a mean evaluation of 4.92 out of 10 is reported, less than the middle ranking value of five. The management of infectious diseases and extraordinary circumstances–particularly prevalent aims amidst the COVID-19 pandemic–receive the highest evaluation with 6.27 and 5.98 respectively. While these two evaluations are low in absolute terms, it is noteworthy that they come on top in the context of the study. Participants appear to perceive the system's response during the first and second waves of the pandemic as comparatively better than its serving of long-term aims during times of normalcy. Furthermore, in line with the disproportionate focus of the policy community toward the care dimensions of public health, the application of scientific achievements in medicine for the protection of public health and the prevention and promotion of health are the two only other aims with a mean score over 5, receiving 5.82 and 5.08 respectively. Monitoring and measuring the population's quality of health, setting aims for the population's quality of health and measuring and improving the effectiveness of health services are evaluated with 4.54, 4.23, and 4.14 on average. The centralized nature of instruments, the low importance attributed to quantifiable aims and the lack of processes for evaluation and human capital improvement, as identified in the previous sections of the analysis, link directly to the poor performance in these areas.

Figure 10. Public health system aims: degree of fulfillment–mean stakeholder evaluations.

Last, the assessment of socio-economic determinants and the containment of health inequalities rank last with mean evaluations of 4.12 each. This result is particularly interesting. Despite an impressively broad provision of processes aimed at social and economic dimensions, these areas remain the less tackled by the system's operation. Adding to the finding is the high importance attributed to socio-economic determinants by stakeholders during the assessment of the macro-context. The holistic approach to health appears to be accepted but not internalized by the system. Rather, parameters beyond services are viewed as exosystemic. Ultimately, the Greek public health system experiences an unquestionable a divide among its components within the health sector and the rest; with the latter receiving low levels of both recognition and investment. Meanwhile, health inequalities have been one of the dominant issues of Greek healthcare since the 1980's, when health policy was contained to just public hospital services (Sissouras, 2012). While the principles of public health policymaking have promoted an extended focus, beyond secondary care, to address the problem more effectively, suboptimalities are proliferating.

Overall, according to the policy community's participants, the paradigm which the system is reinforcing is strikingly different to the one it is supposed to (Figure 11). When asked to evaluate the degree to which the holsitic approach to health is served by the current functioning of the Greek public health system in a 10—point scale—from 0, not at all, to 10, completely—, respondents gave a mean evaluation of 4.70 out of 10. Moreover, 68% of answers gave a ranking of 0—5 and only 2 of 261 participants submitted scores of 9 and 10 which would signify the full or near-full installment of the holistic approach.

Figure 11. Degree to which the Greek public health system's outcomes serve the holistic approach to health.

It thus becomes evident that the the de facto operation of the Greek public health policymaking system diverges from its de jure design, as recognize its participants. However, crucially, the system's limited, healthcare-centric character is not a surprising development but rather an expected evolution given the preferences and orientation of participating agents and the distribution dynamics of resources and processes. As a result, the findings of the evaluation speak of issues in both the embodiment of aims and the functioning of means.

The impact of the COVID-19 pandemic on the Greek public health system

Revisiting the macro-context, the timing of the evaluation was driven by the need to understand the ramifications brought by the epidemiologically unknown and rapidly spreading virus sweeping the globe since February 2020. While the impact of COVID-19 disruptions has been undoubtedly multifaceted, we were interested in the relative ranking of impacted areas in the Greek case according to the perceptions of our respondents (Figure 12). In a 10-point scale–from 0, not impacted at all to 10, highly impacted–three of the four highest mean evaluations include areas which the Greek public health system is primarily responsible for. In specific, the impact of the pandemic on mental health is evaluated with 7.90 out of 10 by stakeholders, on the management of chronic diseases with 7.81 and on the access to health services not related to COVID-19 with 7.70. Meanwhile, working and educational conditions—whose impact on the population's quality of health is also provisioned to fall under the management of the system—rank third with 7.80. Income (7.63), social interactions (7.46), non-occupational activities (6.43) and the family environment (6.23) follow, with the total mean evaluation of 7.37 highlighting that the pandemic's impact has been significant across domains.

Figure 12. Impact of the COVID-19 pandemic across domains–mean stakeholder evaluations.

Amidst dealing with the threat, various emergency processes have been adopted by the system. Stakeholders were called to evaluate their contribution on a 10-point scale, from 0, no contribution to 10, maximum contribution (Figure 13). Overall, all processes receive a mean evaluation of over 5 out of 10, with the aggregate mean of 7.01. Most highly ranked are the social distancing measures, receiving a mean score of 8. Struggling with decentralization, an all-encompassing solution was likely an imposed course of action for the Greek public health system. As a result, the very positive evaluation of lockdown policies likely stems both from a pragmatic assessment regarding the covering of deficiencies and from an evaluation of outcomes. During the first wave of the pandemic, Greece managed to contain the spread of the virus to impressively low levels. In the period during which the first lockdown was imposed–between the 23^rd of March and the 4^th of May 2021–daily cases never exceeded 156 and daily deaths never exceeded 10 (National Public Health Organization (Greece), 2020). Meanwhile, Greece's Mediterranean neighbors were experiencing constant uncontrolled spreading, with Italy surpassing 5,000 daily cases and 900 daily deaths during the same period (Our World in Data, 2020).

Figure 13. Contribution of processes to the management of COVID-19–mean stakeholder evaluations.

Following social distancing measures, respondents ranked the information and communication strategy with 7.49, the hospital units' transformation with 7.23 and the contribution of the National Public Health Organization with 7.21. The awareness and guidelines communication campaign found surprising levels of public appeal, especially at the beginning, with the popularity of the country's chief epidemiologist Sotiris Tsiodras skyrocketing to 94.6% (Ta Nea, 2020). As a result, we were probed to explore informational resources further, with conclusions presented in the following subsection. Transformations in hospital units were among the immediate pressing necessities, as without a vaccine or dedicated medicine, the management of the COVID-19 disease left no middle ground between prevention and secondary care treatment. Long-term underfunding in hospitals had left the number of ICU beds at a mere 560 in the beginning of the pandemic (Psaropoulos, 2020), 5.2 per 100,000 population, less than half of the OECD average of 12 beds per 100,000 (OECD, 2020). By May 2020, beds had increased to 1,017, putting the country almost on par with the EU average (Athens News Agency., 2020). Finally, the National Public Health Organization has been the rebranded version of the Greek Center for Disease Control and Prevention since 2019, coming into its current form since the 2020 public health legislative revision (bill 4675/2020). Although the agency was not assigned major policy steering responsibilities, it handled the digital dissemination of protective guidelines. Overall, its functioning was positively evaluated by respondents.

Processes beyond the national central level were ranked lower. The application of WHO and EU recommendations was evaluated on average with 6.89 and 6.74 respectively, teasing delays and deficiencies in coordination on the part of international organizations and highlighting the strong national character of the pandemic response (Zahariadis and Karokis-Mavrikos, 2022). Similarly, strengthening primary care was ranked last with a mean evaluation of 5.91. The identified lack of regional and local instruments in the system prevented primary care from emerging as an important pillar in the response, while there was little initiative from policymakers for its strengthening despite struggles with the timely containment of clusters after the Summer of 2020. Last, economic measures received a mean score of 6.57 out of 10, pointing to the great COVID-19 management dilemma faced by governments across the globe. Higher financial alleviation could incentivize the population to stay at home and stay safe, but it would simultaneously risk unsustainable future fiscal burden, especially for economically plagued countries like Greece.

All in all, excluding the beyond-expectations successful information and communication strategy, the most highly contributing processes during the COVID-19 pandemic response were the ones which managed to cover the simultaneous exposing of the various deficiencies of the system. On one hand, this denotes good alarm reflexes but on the other, it points out that despite provisions, the system's focus is limited and highly centralized. This confirms the previous conclusions of the analysis and generates calls for adjustment rather than mere masking.

Last, given the widely recognized success in communicating intrinsically scientific information and guidance, the ranking of informational resources by respondents proves essential to best evaluate the system's coping with the COVID-19 threat (Figure 14). When asked to select the three major sources delivering COVID-19 guidelines to the public during the period of study, 73.56% of stakeholders selected the scientific community and 62.84% selected the National Public Health Organization, placing them in the first and third place respectively. Both results offer important insight. From the overall evaluation, it emerged that while the value of scientific inputs has been highly recognized, their systemic usage through resources and processes has either been limited in scope or undermined. The emergence—and acceptance—of science as a steering force amidst the highly centralized and politicized pandemic response highlights that when meaningfully listened to, evidence-based recommendations are undoubtedly essential for the containment of public health threats. Meanwhile, they can increase the support for policy options both among the public and the policy community. Moreover, it is noteworthy that the scientific community ranks higher than the designated public health scientific instrument, the National Public Health Organization. The pandemic tested administrative capacity and highlighted the essentiality of dedicated public health expertise. The National Public Health Organization, despite limited responsibilities, had significant impact. While it has been concluded that scientific sources in the Greek public health system remain medicinally-oriented at large, public health specialization is needed and is irreplaceable. Administrative adequacy is not there yet, but the system calls for it to develop. Last, it is important to underline that the top-four options selected by respondents are official sources, suggesting that the containment of potential misinformation played a major role in the success of the communication campaign.

Figure 14. Sources of COVID-19 guidelines and information.

The overall evaluation–Discussion

In this paper, we set out to revisit the public health system evaluation framework. Our aim was to operationalize its components, test it to the case of Greece and produce insights on the Greek public health system's functioning amidst the ongoing COVID-19 pandemic. Informing our study were 261 stakeholders from the Greek public health policy community, as we sought to understand the de facto reality of the system through the eyes of its operating agents; the supports in the black-box-like systemic model.

First, we operationalized the system's mission and purpose in a twofold fashion. On the one hand, we identified the system's de jure set of aims as described by relevant legislation since 2003. Overall, the Greek public health system's mission is large in scope and aligned with modern developments in public health thought. It ranges from traditional goals such as monitoring, prevention and infectious diseases management to assessing socio-economic determinants and containing inequalities. At the same time, the management of crises including pandemics is set as an explicit aim. On the other hand, we identified the ideational foundation underpinning the system. While the legislative framework puts evaluating health needs and dealing with public health threats at the top of components defining the nature of public health policymaking, participating stakeholders maintain a more care-oriented perspective, focused on prevention, primary and secondary care.

Second, we identified the Greek public health system's functions, a concept we used as a proxy for the grouping of its various processes. We argue that this approach is more meaningful to the evaluation, as the Greek system's functions resemble but also diverge from the 10 essential public health services presented in the original iteration of the framework. From a legislative standpoint, we noted the absence of dedicated processes for the continuous development of public health expertise but recognized the presence of provisions for managing the impact of employment, working conditions and the environment; impactful social determinants, as identified in the system's mission. Nevertheless, through the insights of respondents it emerged that the recognized public health functions which the system works toward are much narrower in focus, as they mostly include vaccination campaigns, eliminating health inequalities, quality-checking food and drink, promoting healthy behavioral patterns and dealing with extraordinary public health threats.

Third, we evaluated resources. As a starting point, we focused on supervising responsibility, a previously unincluded element in the framework's applications. Once again, we noticed discrepancy between provisions and the functioning reality as described by participants. Despite having recognized the importance of regional and local instruments, the Greek public health system relies very heavily on the central government. At the same time, the importance of individual responsibility is increasing–in line with recent policy directions–while non-state actors are minimally recognized. Furthermore, among systemic resources, funding is identified as the most contributing, followed by scientific justification, human capital and management. Crucially, both in the legislative framework and in the perception of stakeholders, quantifiable aims do not receive the same level of attention. Last, among scientific informational resources, medicine, epidemiology and health economics are identified as most valuable, with statistics and social, communication and psychological sciences being rated significantly less.

All the above explain the system's performance as reflected in the evaluated completion of aims. Containing health inequalities, assessing socio-economic determinants, measuring and improving the effectiveness of health services, setting aims for the population's quality of health and monitoring and measuring the population's quality of health all receive mean evaluations below 5 on an ascending 10-point scale, denoting strikingly subpar performance. Through the analytical guidance of our model, the emerging explanation emerges as follows: In a system where the ideational understanding of its purpose among operating agents is more limited than its provisioned mission, processes are disproportionately developed, relevant resources become narrow in scope and outcomes are suboptimal. Meanwhile, high centralization and politicization and the absence of dedicated public health expertise through administrative instruments prevents the system from understanding its failures and adjusting its course. Ultimately, the Greek public health system is experiencing a two-gear problem. First, the provision of services is pursued endosystemically, but suffers from the absence of fiscal and organizational resources. Second the containment of quality-of-health threats is largely viewed as an important, but external to the system, parameter, hence falling victim to jurisdictional voidness and macropolitical impediments. This latter dimension is highly intertwined with prevailing stakeholder perceptions, which acknowledge but haven't yet embraced the holistic perspective to health in the functioning of the system, confirming H1.

Undoubtedly, the evaluation strongly reflects the testing of the system by the ongoing pandemic. Our approach sought to capture the COVID-19 pandemic's effect in both a latent and explicit fashion. Among the general evaluation of respondents, there is a clear divide between the assessment of managing infectious diseases and containing public health crises and the rest of the system's aims and functions. While the intense lockdown policies allowed Greece to cope with the COVID-19 threat, at least during the first wave, they also cemented in the eyes of stakeholders that the system is suffering across the board. Moreover, the top-down character of the response fundamentally proved that public health in Greece is predominantly driven by the actions, guidance and supervision of the central government. From an explicit evaluative standpoint, COVID-19 challenged first and foremost areas in which the public health system already has primary responsibility, confirming H2. At large, the legislative design appears to have gotten the system's scope right; it has just failed to establish comprehensive processes and to develop strong supporting resources, hence allowing problems to manifest and propagate over time.

Building-up undermined processes in the face of crisis proved utopian. For example, the contribution of a strengthened primary care was evaluated significantly less that social distancing measures. This is why the overall evaluative insights are crucial. Policy change during conditions of crisis is more likely to emerge in the form of institutional bricolage–i.e., adding or recombining capacity ad hoc—(Mahoney and Thelen, 2010) rather than in the form of repurposing. Last, the demands of the threat induced a sorting of resources that brought forward the flaws of long-standing misdirections. The contributions of the scientific community and of the National Public Health Organization to the guidelines and communications strategy highlighted in strong fashion the essentiality of scientific inputs and dedicated public health expertise.

We present a visualization of the conclusions below (figure 15).

Figure 15. Public health system performance evaluation: a refined model, the case of Greece.

Conclusions, policy insights and future research directions

Evaluating public health systems in the face of a public health emergency speaks to the heart of a broader problem: policy evaluation is underappreciated, and policymakers find themselves scraping for adjustments in the face of crises. Among the aims and functions of the Greek public health system, consistent evaluation and feedback inputs are nowhere to be found. An innovative policy design does little in practice if it is not followed by comprehensive provisions that will allow the establishment and reproduction of the paradigm it promotes. To this end, the insights of the analysis offer both policy and future research directions.

For Greece, the COVID-19 pandemic brought forward long-standing problems, which, crucially, are not entirely novel and have largely sought to be addressed by public health bills since 2003. At its current stage the system demands five essential adjustments to pursue successful functioning: (1) A repurposing of aims to unquestionably define its scope and its participating institutions and instruments, (2) meaningful and powerful administrative instruments, especially beyond the central level, with decreased politicization in their staffing and functioning, (3) the fostering of public health expertise and the higher reliance on it for inputs, implementation and evaluation, (4) more detailed processes, orientating the frame of interaction between the system's components and the measurable aims they should deliver and (5) consistent attention to the system, not just in the face of pandemics. As long as the system's stakeholders–at large, the respondents in our survey–pursue the fifth, the other four can develop when public policymaking conditions are favorable.

From a public health scholarship perspective, the evaluative approach employed can hopefully steer the development of wider research agendas. Public health has well and truly returned to the forefront since 2020. As much as it is a challenge, COVID-19 is also an opportunity; much like the evolutions of the 1980's and 1990's. Extending the focus of evaluation past the metrics of effectiveness, efficiency and equity, can produce invaluable insights for the future design and refining of public health systems. As a whole, our operationalization intends to instigate the revisiting of what goes into the evaluation of public health systems. We acknowledge that our contribution is the product of a single case-study perspective, focused on aggregate insights and that data collection can and should explore systemic components in even more depth in the future. However, we posit that stakeholder perceptions can provide a metric of true operational understanding, past the disguise of institutional mapping. If complemented with the larger sets of data quantifying the assessment of component performance, research programs can break new policy ground. Alas, what has the pandemic taught us if not that public health is well and truly the exercise of public policy?

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by National and Kapodistrian University of Athens, Department of Politics. The patients/participants provided their written informed consent to participate in this study.

Author contributions

VK-M: development of model and data processing. MM: survey design, data collection, and development of model. JY: sampling and survey design. All authors contributed to the article and approved the submitted version.

Conflict of interest

Author MM is also employed by Vianex S.A. (Athens, Greece).

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpos.2022.1067250/full#supplementary-material

Footnotes

References