https://orcid.org/0000-0001-7786-5342
Figures
Abstract
The COVID-19 outbreak has highlighted our vulnerability to novel infections.
Faced with this threat and no effective treatment, in line with many other countries, the UK adopted enforced social distancing (lockdown) to reduce transmission—successfully reducing the reproductive number R below one. However, given the large pool of susceptible individuals that remain, complete relaxation of controls is likely to generate a substantial further outbreak. Vaccination remains the only foreseeable means of both containing the infection and returning to normal interactions and behaviour. Here, we consider the optimal targeting of vaccination within the UK, with the aim of minimising future deaths or quality adjusted life year (QALY) losses. We show that, for a range of assumptions on the action and efficacy of the vaccine, targeting older age groups first is optimal and may be sufficient to stem the epidemic if the vaccine prevents transmission as well as disease.
Author summary
In line with most other countries across the globe, and in the absence of a vaccine or pharmaceutical treatments, the UK has relied heavily upon non-pharmaceutical social measures to control the impact of the COVID-19 pandemic. While this has proved effective in reducing the healthcare burden compared to an uncontrolled outbreak, this is achieved to the detriment of the economy, education and many other societal factors. As vaccines are developed which mitigate the disease, it is of great importance that they are delivered in an optimal manner—reducing mortality and healthcare demands. Using an age-structured mathematical model of SARS-CoV-2 transmission, we test different vaccine ordering strategies to identify which members of society should be targeted for vaccination first in order to achieve a specified health objective. In all scenarios we find vaccinating the most elderly and vulnerable first to have the greatest impact, though the ultimate success of any vaccination scheme will be highly contingent on the characteristics of the vaccine itself and the level of population uptake.
Citation: Moore S, Hill EM, Dyson L, Tildesley MJ, Keeling MJ (2021) Modelling optimal vaccination strategy for SARS-CoV-2 in the UK. PLoS Comput Biol 17(5): e1008849. https://doi.org/10.1371/journal.pcbi.1008849
Editor: Alex Perkins, University of Notre Dame, UNITED STATES
Received: October 9, 2020; Accepted: March 3, 2021; Published: May 6, 2021
Copyright: © 2021 Moore et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data on cases were obtained from the COVID-19 Hospitalisation in England Surveillance System (CHESS) data set that collects detailed data on patients infected with COVID-19. Data on COVID-19 deaths were obtained from Public Health England. These data contain confidential information, with public data deposition non-permissible for socioeconomic reasons. The CHESS data resides with the National Health Service (ww.nhs.gov.uk) whilst the death data are available from Public Health England (www.phe.gov.uk). Interested readers wanting similar data may wish to contact coronavirus-tracker@phe.gov.uk.
Funding: This report is independent research funded by the National Institute for Health Research (NIHR, https://www.nihr.ac.uk/) [Policy Research Programme, Mathematical Economic Modelling for Vaccination and Immunisation Evaluation, and Emergency Response; NIHR200411]. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. It has also been supported by the Engineering and Physical Sciences Research Council (https://epsrc.ukri.org/) through the MathSys CD [grant number EP/S022244/1] and by the Medical Research Council through the COVID-19 Rapid Response Rolling Call [grant number MR/V009761/1]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
After its initial detection in late 2019, the SARS-CoV-2 virus has spread across the globe with more than 20 million cases detected by mid-2020 [1]. For many individuals, infection develops into COVID-19 disease, with symptoms including fever, shortness of breath and altered sense of taste and smell, potentially escalating to a more severe state which may include pneumonia, sepsis, and kidney failure [2]. In general these symptoms and their severity have been observed to increase with age, such that it is the elderly that experience the greatest burden of disease. Due to the novelty of the virus, rapidity of its spread and the lack of effective treatments, mortalities (again predominantly in the older age-groups) have also been considerable.
By August 2020, six months after the initial cases in the UK, the country has reported in excess of 314,000 cases and more than 40,000 COVID-19 deaths [1]. As in many other countries, the virus spread in the UK has been mitigated by the introduction of a range of social distancing measures including the closing of workplaces, schools, pubs and restaurants, and the restriction of a range of leisure activities. Due to the continued risk of a considerable further infection, combined with the negative social and economic impact of continued social-distancing measures, a vaccine is urgently sought that may help curtail the global pandemic.
Mathematical models of COVID-19 dynamics and the impact of vaccination are important for public health planning and resource allocation. However, a substantial challenge of modelling vaccination against SARS-CoV-2 at this stage is the large number of unknown factors. At the time of writing, the UK has reportedly made deals for six different vaccine candidates created via differing approaches [3]: the Oxford (ChAdOx1 nCoV-19) [4, 5] and Janssen (Ad26.COV2.S) [6] vaccines, made from a genetically engineered virus; a vaccine developed by BioNtech/Pfizer (BNT162b1) [7], which uses an novel approach of injecting part of the virus’ genetic code; a vaccine created by Valneva (VLA2001) [8], which uses an inactive version of the virus; a vaccine created by Novavax (NVX-CoV2373) [9] and one under development by GlaxoSmithKline/Sanofi Pasteur [10], both using protein adjuvants to stimulate an immune response.
At the time of writing, with an array of possibilities, differing in mechanism and still in varying stages of development, estimating response characteristics was infeasible. While the ideal vaccine would work to decrease susceptibility [11], thus limiting viral spread, the first successful candidate may just limit the occurrence or severity of symptoms. Moreover, it was still unclear when any vaccine would reach the final stage of being ready for mass deployment. Finally, while some promise had been shown in small human and animal studies—the success of the Oxford vaccine in a trial involving Rhesus Macaques [7] generated considerable excitement for instance—robust evidence as to how any vaccine candidate would perform in the wider human population was lacking [12].
Due to this uncertainty, we utilise a previously developed SARS-CoV-2 transmission model [13, 14] to understand the likely dynamics following the deployment of a vaccine. Rather than testing a single specific vaccine product, we assess three types of vaccines throughout our analysis to produce an evaluation that is as informative as possible for any likely set of vaccine characteristics. Explicitly, these comprise of: a vaccine that reduces susceptibility, thus being effective in inhibiting viral transmission as well as protecting the individual; one that reduces the probability of becoming symptomatic, which still has some benefit in reducing transmission as the model implicitly assumes that transmission from asymptomatic infections is less than from symptomatic cases (in agreement with observational studies that have found asymptomatic cases of COVID-19 to be much less likely to infect others than symptomatic cases [15]); and one that protects against symptoms becoming severe, providing the direct protection against disease to the vaccinated individual only.
For each vaccine type we predict the impact of a range of possible scenarios by analysing sensitivity to vaccine efficacy, which may be uniform or age dependent, as well as the scale and targeting of deployment. Further, by considering different orders of prioritisation in terms of age group and health conditions, we arrive at an optimal targeting strategy for vaccination that achieves the greatest reduction in disease impact for the number of doses administered.
For a variety of other diseases, there is a precedent for combining modelling approaches with health economic evaluations to inform vaccine policy decisions based on a willingness to pay for each Quality Adjusted Life Year (QALY) saved [16–19]. Utilising this framework, we also consider how vaccination may be optimised to minimise the loss in QALYs (resulting from hospitalisation and mortality as a direct consequence of COVID-19), rather than simply the number of deaths. This methodology could allow a monetary value to be assigned to each dose of vaccine (for the calculation of QALYs lost, see Section 5 of the S1 Text); while some cost benefit analysis has been pursued in relation to SARS-CoV-2 [20, 21], the unprecedented scale of the pandemic invalidates the usual metrics in this approach. We therefore do not utilise this health-economic approach as the main focus of our paper.
Despite the obvious need for both country-specific and more generic forecasts of the effects of vaccination on the COVID-19 outbreak, before this study it had received relatively limited attention [22–24]. Our study adds a UK specific perspective to these studies and generates a wider view on how the type of protection offered by the vaccine as well as the underlying efficacy interact with targeting risk-groups to generate maximal benefit.
Methods
Ethics statement
The data were supplied from the CHESS database after anonymisation under strict data protection protocols agreed between the University of Warwick and Public Health England. The ethics of the use of these data for these purposes was agreed by Public Health England with the Government’s SPI-M(O) / SAGE committees.
SARS-CoV-2 transmission model
We used a compartmental age-structured model, developed to simulate the spread of SARS-CoV-2 within regions of the UK [13], with parameters inferred to generate a good match to deaths, hospitalisations, hospital occupancy and serological testing [14].
It involves an extended SEIR-type framework: susceptibles (S) may become infected and move into a latent exposed (E) state before progressing to become infectious. Echoing the observed behaviour of COVID-19 infections, the model differentiates between individuals who are symptomatic (D, likely to be detected) and those who are asymptomatic (U, likely to remain undetected). Formulated as a system of ordinary differential equations (listed in Section 1 of the S1 Text), model simulations involved numerically integrating these sets of equations.
Partitioning those infectious by symptom status allows for the lower level of transmission believed to be associated with asymptomatic infection. It also generates the possible progression of symptoms increasing in severity, leading to hospitalisation and/or death (Figs 1 and 2, see Section 2 of the S1 Text for additional information on computation of hospitalisation and mortality quantities).
Descriptions of model parameters are given in Fig 2. Individuals starting from the S (susceptible-unvaccinated) or V (susceptible-vaccinated) states move through the model to conclude in the R (recovered) or Rd (dead) states. Those in the asymptomatic, U, state all eventually recover, those in the symptomatic, D, state may either recover or die directly, or move into hospital and/or ICU before either recovery or death. Three different possible types of vaccination are shown by green arrows and bracketed variables. These are:
- Reduction in susceptibility.
- Reduction in becoming symptomatic.
- Reduction in experiencing severe symptoms.
Further model details may be found in the supporting information.
The model is stratified by age structure, with force of infection determined by the use of age-dependent (who acquires infection from whom) social contact matrices for the UK [25, 26]. Additionally, we assumed susceptibility and the probabilities of becoming symptomatic, being hospitalised and dying to be age dependent, and are matched to UK outbreak data. Finally, our model formulation accounted for the role of household isolation by allowing first infections within a household to cause new secondary infections at an increased rate (more details may be found in [13]). This allows secondary household contacts to be isolated and consequently play no further role in the outbreak (further details are provided in Section 1 of the S1 Text). Model parameters were inferred on a regional basis using local time series of recorded daily hospitalisation numbers, hospital bed occupancy, ICU occupancy and daily deaths [14] (see Section 3 of the S1 Text).
This model formulation is then extended to capture a range of vaccination scenarios. Below we detail how vaccination of different forms is introduced into the model; the degree to which non-pharmaceutical interventions are reduced; and how the population is partitioned according to high-risk comorbidities—to assess if these are a priority group for immunisation.
Comorbidity
Alongside age, it is well understood that underlying health conditions are critically important in determining the likelihood of individuals to experience severe symptoms; in younger, healthier populations a relatively low symptomatic rate has been observed [27–30] and there are few severe cases. To account for heterogeneity of risk that is not attributable to age, we therefore allowed the probability of experiencing severe health outcomes (as a result of COVID-19 disease) to be dependent on underlying health conditions.
We calculated the additional risk by comparing the prevalence of conditions amongst COVID-19 mortalities within the general population. For the purpose of simplification, we used a binary system to divide the population into those with significant health conditions and those without, rather than considering individual conditions separately [31].
We estimated the proportion of the population at heightened risk, due to the presence of a cormobidity, using the twelve health conditions with the highest associated risk factors as identified in a recent study by Williamson et al. [32]. We summarise these conditions, alongside their prevalence and individual risk, in Table 1.
For each condition we give the estimated prevalence in the population as well as the increased risk of death found as a hazard ratio calculation adjusted for demographics and coexisting conditions.
For the 18.42% of the population with one or more of the comorbidity conditions listed in Table 1, we calculated that the average increase in the risk of morbidity is 2.43 (mean value taking into account condition dependence)—suggesting that individuals with these risk factors are more than twice as likely to die from COVID-19 infection compared to others of the same age. The distribution of these combined symptoms by age group was calculated using data from the Royal College of General Practitioners Research and Surveillance Centre [33] combined with statistics on cancer and diabetes prevalence found in [34, 35].
The comorbidities associated with increased risk from COVID-19 infection are found to occur predominantly in elder age groups (Fig 3a), with individuals above 75 years of age more likely than not to have underlying health problems. When these factors are incorporated into the predictive model, we see that deaths are dominated by older age groups and individuals with underlying comorbidities (Fig 3b).
The proportions of each age group with health conditions are shown in red, with the proportion of each age group without health conditions (healthy) shown in purple.
Social measures
The use of non-pharmaceutical social-distancing (or lockdown) measures has already proven to have substantially reduced the scale of the pandemic [36]. We include such measures into our model through a reduction in the relevant contact matrices (i.e. scaling down matrices associated with schools, work places and other settings) whilst increasing the level of contact within household environments [13, 14] (expanded details are given in Section 4 of the S1 Text). Additionally, we incorporate household quarantining by transferring a proportion of the symptomatic individuals to a quarantined state with a still greater reduction in external contact.
It is expected that a sustained level of social-distancing measures will continue to be used to mitigate the disease until pharmaceutical approaches are available. As such, in all simulations, we initially simulate the first wave of the UK epidemic, followed by a continuation of lock-down measures (generating an initial R ≈ 1.0) until vaccination is completed.
As exemplified by simple models [37, 38], the impact of any vaccination campaign is highly dependent upon both the proportion of the population successfully immunised and the reproductive ratio. As such, we expect the success of any COVID-19 vaccine programme to depend on the reproductive ratio R when the programme begins, which in turn is dependent upon the level of non-pharmaceutical interventions (NPIs) and the proportion of the population already infected. Fig 4 compares simulations in which all NPIs are lifted at the start of 2021 (generating an initial R ≈ 2.3 at this point), with a scenario in which a limited number of NPIs are still in place (leading to R ≈ 1.8).
(a) First wave of infection as observed from the start of 2020. (b) Subsequent wave of infection following relaxation to limited NPIs, sufficient to reduce the basic reproductive number to approximately R = 1.8(±0.1). (c) Subsequent infection wave following complete relaxation of all NPIs leading to R = 2.3(±0.2). Uncertainty is represented by the shaded region, within which 95% of simulations are found to fall.
Given the general level of awareness and concern in the population, we take R ≈ 1.8 as our base-case during the majority of our simulations, but do assess a return to pre-COVID behaviour (R ≈ 2.3) on completion of the vaccination campaign. To tune the initial reproduction number R to the desired value, the scaling of the contact arrays was set to obtain a specified growth rate r (computed by an eigenvalue approach), corresponding to the requested R value.
Vaccination
The novelty of the virus, combined with the urgency in finding a response, has resulted in a plethora of possible vaccine candidates in various stages of development across the globe [39, 40]. To account for uncertainty in both the type and the degree of protection a vaccine may provide, we test the effects of three different vaccination mechanisms within our model, each over a range of efficacy:
- Reduction in susceptibility (type 1): We reduce the rate at which the vaccinated susceptibles may become infected. This is the most effective form of vaccine as it may significantly reduce R as well as protecting the individual.
- Reduction in becoming symptomatic (type 2): The age and health dependent probabilities of becoming symptomatic are adjusted according to vaccine efficacy. This type of vaccine therefore has a substantial impact on disease, but also has some impact on infection spread (and hence R) due to asymptomatic individuals being less infectious.
- Reduction in experiencing severe symptoms (type 3): The age dependent probabilities of symptoms becoming severe (leading to hospitalisation and/or death) are adjusted according to vaccine efficacy. This type of vaccine only protects the individual and does not impact the transmission dynamics.
The action of these three vaccination types are shown in Fig 1. It should be noted that, whilst instantaneous vaccination may not be a literal possibility, due to the uncertainty on time scales combined with the potential for further infection to be delayed by social-distancing measures during vaccine deployment, it is a reasonable modelling assumption for our exploratory purposes. Similarly, to constrain the dimensions of our exploratory analysis we assumed no loss of vaccine-induced immunity for the duration of the simulations (with a similar assumption applied to immunity derived from natural infection).
Simulation specifics
In all simulations, we ran the model for an initial period (until the end of 2020) without any form of vaccination, to create a base-line population of susceptible, infectious and recovered (immune) individuals that approximates the state of population when vaccination begins. Given that during this period R is close to, but below one, the precise timing of the start of the vaccination programme has little effect. Similarly, if NPIs are maintained until vaccination is complete, the speed of vaccination is unlikely to play a major role.
At the start of 2021, to simulate rapid immunisation, we transfer a specified proportion of the remaining susceptible individuals into a vaccinated state; although both infected or recovered individuals may also be vaccinated this does not impact their dynamics and can be ignored. The action of infection on individuals in the vaccinated class is dependent on vaccine type.
We now outline the four/five different approaches we use to quantify the optimal targeting of vaccine and the sensitivity to key uncertainties.
Priority order of vaccination.
To determine a vaccination priority order for who should receive the vaccine first, we ranked each group by the ratio of the number of doses administered to the reduction in the outcome measure of interest. Our two selected outcome measures were: (i) deaths post-2020; (ii) quality adjusted life years (QALYs) lost post-2020. We divided the population into 20 year age bands (0-19,20-39,40-59,60-79,80+), together with an additional group of people with comorbidities independent of age. For each vaccination scenario considered, we exhaustively tested the order of deployment amongst these groups.
Prioritisation of healthcare workers.
Due to their significantly heightened exposure, another group that may benefit from early targeted vaccination is healthcare workers (HCWs). Proportionally, 11.6 times more cases have been observed in UK HCWs than the general populace—this can partially be explained by the increased testing in hospital environments. Yet, even accounting for this increased testing, HCWs are estimated to experience a 3.40 (3.37-3.43) fold increase in infection risk [41].
We included this increased risk to HCWs in our model by adjustment of the susceptibility parameters, using a similar methodology that was applied to underlying health conditions. This increased susceptibility was applied to all of the 1,134,824 full time NHS employees [42], which we assumed to be evenly distributed (age and region proportional) amongst the working age group (20-64).
Sensitivity to vaccine characteristics.
Using the identified optimal prioritisation order for the given vaccine type, with a maximum vaccine uptake of 70% across each sub-group in the population, we analyse the sensitivity of number of deaths and QALYs lost to vaccine efficacy, considering vaccines with 50%, 70% and 90% efficacy. Here we define an efficacy of n% to give an n% reduction in susceptibility/chance of becoming symptomatic/chance of developing severe symptoms for a type 1/2/3 vaccine respectively.
Sensitivity to speed of vaccine deployment.
In the majority of the modelling we assume that the time needed to vaccinate the population and for the vaccine to confer immunity is short. This assumption has little impact if the reproduction number R is below one while the vaccine program is being undertaken.
To explore the impact of slow vaccine deployment in the face of increasing infection we utilise simulations in which NPIs are relaxed, to bring R up to 1.8, two months before a type 1 vaccine with 70% efficacy begins deployment, thus delivery occurs alongside an already widespread and increasing epidemic. We simulate gradual delivery dynamics by running our model over daily time steps with fixed number of susceptible or asymptomatic individuals being successfully immunised at each interval. More realistic delivery scenarios may be simulated as vaccination deployment plans are better understood.
Sensitivity to age-dependent efficacy.
Many existing vaccines are not uniformly effective and response may vary significantly due to factors including age [43]. In the case of a virus where severity of symptoms are age-dependent, like SARS-CoV-2, age-dependent vaccine efficacy may have a significant impact. Of particular concern is whether an age-dependent decline in efficacy could alter the group priority order for receiving a vaccine, and the potential scale of the subsequent outbreak. We test the likely implications of reduced vaccine effectiveness in the elderly by considering an array of different efficacy profiles consisting of a base efficacy uniform amongst those below the age of 45 with linearly declining efficacy to a minimal value shared by those above the age of 85.
For each of the scenarios described above and for each combination of considered vaccine type, efficacy and priority order, we performed 100 separate simulations. Each simulation used a distinct parameter set drawn from posterior parameter distributions, which were obtained from the inference procedure that tuned the model to the available COVID-19 health outcome data streams for the UK [14].
Results
We now consider the ordering of vaccination and its impact in terms of the number of deaths and associated QALYs lost. For the majority of the situations we consider, we plot the number of individuals vaccinated on the x-axis (assuming a maximum of 70% of susceptible or asymptomatically infected individuals are vaccinated in any group) and the resultant number of deaths or QALY losses on the y-axis. The priority ordering is determined by which groups generate the greatest reduction for the number of doses administered (steepest decline on the graphs).
Selection of priority order
In every case, optimising for either a reduction in death or a reduction in QALYs was found to give consistent ordering, due to the substantive contribution of mortality events to the overall QALY loss (Fig 5). When structuring by age alone, the most efficacious reduction was found through an oldest first approach—despite not being the most crucial group in terms of transmission, the considerably heightened vulnerability amongst the elderly means that priority should be given to protecting them directly. We also highlight the substantial advantages of a strategically targeted vaccine over an unbiased (random) approach in every scenario (blue line, Fig 5).
In each panel, the simulated points represent the number of subsequent deaths expected after vaccinating everyone up to a given group following an optimum strategy (identified by giving greatest reduction in deaths per vaccination in each instance) represented by the purple and orange connecting lines. Here the purple line shows the path of the optimum vaccination ordering of groups comprising of 20 year age bands together with comorbidities across all ages, and the orange line for age groupings only. The blue lines show an unbiased strategy with vaccinated numbers dispersed evenly across the population and the grey dotted line show the base level of morbidity from the first infection wave.
In contrast, there was less consistency in the optimal position of vaccinating comorbidities in the priority order, which varied between just before and just after the 60-80 age group. One such instance in which the discrepancy may be seen is when contrasting the type 1 and type 3 vaccine with a low level of NPIs (R ≈ 1.8, Fig 5a and 5b). As may be seen in Fig 3 there is a strong correlation between age and prevalence of comorbidities meaning that an age only approach is already relatively effective in targeting the most vulnerable first. The significance of the comorbidity group is also reduced by fact it spans age groups itself (the ideal program would involve an ordering of each age group split into those with and without comorbidities though in practice this would likely be too complex to implement). The relatively small cost-benefit implications we see by varying the priority of the group of people with comorbidities, due to this, may still be substantial enough however to be worth critical consideration in a scenario where vaccine quantity and/or deployment rate are limited.
Healthcare workers
We now consider the addition of healthcare workers (HCW) as an additional risk group, and assess their optimal position in the order of vaccination priority. For a type 1 vaccine (for which there is benefit in targeting infection spreaders as well as the vulnerable) we associate high important to the vaccination of HCW, second only to the most elderly age group of those aged 80 and above (Fig 6). It is important to note that our treatment of HCWs only accounts for increased personal risk. We have not taken into account the increased contact between HCWs and the vulnerable which may significantly amplify their risk as spreaders of infection, increasing their priority for vaccination. Given also the relatively small number of HCWs (less than 2% of the population), we conclude that this group should be included as a high priority group for vaccination. However, for vaccines of type 2 or 3 that do not significantly reduce infection spread, the HCW group may be considerably less important and their priority should be judged accordingly.
Vaccine type comparison under optimal priority order
For the three types of vaccine action, and for three different levels of efficacy we consider the impact of a vaccine program targeted by age and comorbidity under weak non-pharmaceutical interventions (R ≈ 1.8). For Type 1 vaccines, which have the greatest impact, we also consider complete lifting of all non-pharmaceutical interventions (R ≈ 2.3).
Type 1 vaccine.
Due to the ability of a type 1 vaccine in preventing the spread of infection as well as protecting specific individuals, we find that such a vaccine, even with relatively low efficacy, could be highly effective in preventing further COVID-19 mortality when combined with limited social-distancing measures (i.e when R ≈ 1.8).
The best performing prioritisation order begins with those aged 80 and above, followed by those with health conditions, before the rest of the population in age order. Under such an ordering, we estimate a vaccine with 50% efficacy delivered to 70% of the population above the age of 20 to be sufficient to prevent a SARS-CoV-2 resurgence in our considered scenario (Fig 7 top row), although a more efficacious vaccine could achieve the same results with only vaccinating those above 40.
We use the optimal vaccination ordering with a maximum vaccine uptake of 70% across the population. The optimal vaccination ordering did not depend on the collection of considered vaccine efficacies. We display the mean values of 100 independent simulations for each vaccine efficacy (solid lines), with 95% prediction intervals (shaded regions). The grey dotted line shows the base level of mortality from the first wave of the pandemic.
Type 2 vaccine.
When combined with limited social-distancing measures (such that R = 1.8), a type 2 vaccine (that reduces the chance of becoming symptomatic) with reasonable efficacy may still be sufficient to prevent a significant further mortality (Fig 7 second row). However, due to only providing a limited reduction in transmission little additional benefit is to be found in vaccinating those in low risk (younger age) categories. Again we identify that it is best to prioritise those above the age of 80, followed by those with health conditions, before the rest of the population in age order. For type 2 vaccines that generate an efficacy of 50% or below we would predict a large-scale subsequent wave of infection when control measures are relaxed with a comparable number of deaths to experienced in the first wave.
Type 3 vaccine.
A type 3 vaccination, which only reduces the chance of a recipient experiencing severe disease, does not reduce the risk of transmission and therefore we predict a large scale second outbreak when containment measures are lifted, comparable to that shown in Fig 4c. However, it can still generate considerable benefit when deployed to high risk individuals, particularly those over the age of 60 (Fig 7 third row). For the most efficacious vaccines, immunising 70% of those over 60 years old could potentially halve the number of deaths following vaccination.
Contrary to the other three vaccination types, here we find it best to prioritise 60-80 year olds above all those with health conditions. We postulate this is due to the lack of reduction in transmission, such that the higher degree of social contact amongst the younger people within the comorbidity group is not an influencing factor.
Type 1 vaccine, without measures.
The most desirable vaccine would be one that is sufficient to entirely contain the pandemic without the need for any other intervention (when we estimate R ≈ 2.3). Amongst our set of vaccine scenarios, we find the only plausible candidate for preventing a SARS-CoV-2 resurgence (in the absence of any other intervention) would be a type 1 vaccine with an efficacy above 80% and with 70% of the entire population vaccinated (Fig 7 bottom row). Other levels of vaccine efficacy would naturally trade-off with a different proportions of the population needing to be vaccinated; similarly a higher uptake in the older at-risk populations would also mean that fewer younger individuals would need to be immunised.
For a type 1 vaccine in a scenario without additional measures (R ≈ 2.3), we find scant difference between taking a fully age ordered strategy, and one including comorbidity, leaving no clear point at which those we categorise as having health conditions should be prioritised (Fig 5).
Slow deployment
The results presented so far are based on a model with instantaneous vaccination delivery. This is deemed sufficient for a generic understanding of the impacts of vaccination, varying uptake levels and targeting. Additionally, if NPIs remain in place (such that R < 1) until vaccination is completed, the time taken to implement a slower vaccination campaign will lead to there being fewer cases when NPIs are finally relaxed. It is also the case that, since temporal factors such as deployment speed and a start date for delivery are were largely unknown before 2021, there was limited value in incorporating complex dynamics at the time we performed these analyses. More realistic delivery scenarios may be simulated as vaccination deployment plans are better understood.
To explore the impact of slow vaccine deployment in the face of increasing infection, we present simulations in which NPIs were relaxed (with R raised up to 1.8) two months before a type 1 vaccine with 70% efficacy began deployment. For the conditions assumed here, the race between vaccination to reach herd-immunity and the rise of the epidemic indicates the speed of vaccine deployment is key (Fig 8); a rapidly deployed untargeted campaign is far more successful than a slow but optimally targeted one. Importantly, we find that the optimal vaccination strategy (in terms of the ordering of age and risk groups) is independent of delivery speed. However, for very rapid vaccine deployment (when all groups can be vaccinated before there are many more infections) or for very slow deployment (when the epidemic is complete before many individuals are immunised) the ordering is far less important.
Vaccine deployment is started 2 months after stricter NPIs are relaxed to leave low level measures sufficient to keep R ≈ 1.8. The purple line (and shaded prediction region) represents projected outcomes under the identified optimal ordering with age and comorbidity groups, the orange line outcomes using the identified optimal ordering accounting for age groups only (without comorbidity), and the blue line outcomes with an unbiased population wide delivery. The grey dotted lines show the base level of mortality or QALYs lost, as applicable, from the first wave of the pandemic in the UK in the first half of 2020.
Age-dependent efficacy
To simulate reduced efficacy of the vaccine in older age groups, we allow efficacy to be age-dependent—attaining a high maximum value for individuals below age 45, then dropping linearly to age 85 when the efficacy reaches a minimum value (Fig 9 left panel inset).
A maximum efficacy value is applied to all individuals below the age of 45 and a minimum level to individuals above the age of 85. Efficacy is assumed to decay linearly between these two levels to give the efficacy for intermediary age groups. This distribution is represented by the inset in left panel. The left panel shows when, dependent on minimum and maximum efficacy, the two groups deemed most significant for vaccination impact vary. In the purple region it is optimal to vaccinate those above the age of 80 followed by comorbidities, in the red comorbidities followed by 80+ and in the yellow those in the 40-60 age group followed by comorbidities. The right panel shows the expected further mortality following vaccination with 70% of the whole population vaccinated for different minimum/maximum type 1 vaccine efficacies. The large dark blue region corresponds to less than 10,000 deaths following vaccination.
There is again a surprising robustness in ordering (Fig 9 left panel) with the elderly (over 80 years old) remaining the key group to initially target, even when the efficacy in this age group drops to just 20%. Even lower efficacies necessitate a switch with individuals with underlying comorbidities or aged 40-60 being prioritised. Lower efficacy vaccines are, unsurprisingly, associated with larger subsequent epidemics and therefore more deaths; much of this increase is concentrated, however, in low efficacy regions of parameter-space, with the estimated number of deaths being relatively constant across a range of efficacy values for both older and younger age-groups (right panel).
While here we focus on a type 1 vaccine; the results for vaccine types 2 and 3 are found to be more extreme, as the drop in efficacy aligns with the age-groups most in need of protection. For a type 3 vaccine the elderly are found to remain the initial targeting priority up to a reduction in efficacy down to just 10%.
Discussion
It has often been stated that the only way for population behaviour to return to normal is through achieving herd-immunity. The large amount of natural infection likely to have occurred by the time a vaccination program may be completed will make a significant contribution towards this, however, allowing this to occur solely through natural infection is far too dangerous, even if the most at-risk groups are shielded, making vaccination the only viable alternative. For SARS-CoV-2, the infection is relatively homogeneous across age-groups with a slight bias towards younger adults [44], but disease—especially severe disease—is associated with old age and a range of comorbidities. As with many vaccines, there is tension between vaccination of those most responsible for driving transmission (vaccination to reduce R) and vaccination of those most likely to suffer severe health outcomes (vaccination to limit disease) [45]. Here, using detailed mathematical models that have been matched to UK COVID-19 outbreak data, we find that vaccine strategies targeting the elderly are optimal in terms of reducing future mortality, even if vaccinating younger group-ages would have a greater impact on the reproductive number, R.
We have found that five main factors influence the success of any vaccination programme (Fig 6).
- The characteristics of the vaccine: vaccines that reduce susceptiblity and therefore prevent onwards transsmission (type 1 vaccines) lead to a far greater reduction in mortality, compared to vaccines that reduce disease (types 2 and 3);
- The efficacy of the vaccine: vaccines of all types with higher efficacy generate both greater protection of individuals and (for type 1 and 2 vaccines) greater protection within the population;
- The reproduction number once the vaccination program is complete: higher reproduction numbers require a greater proportion of the population to be immunised to prevent a second outbreak (mimicking the classic Vc = 1 − 1/R0 paradigm). If all non-pharmaceutical interventions (NPIs) are lifted, and if approximately 70% of the population are vaccinated, only highly efficacious, type 1 vaccines are able to control subsequent waves of infection;
- The proportion of the population vaccinated: throughout we have assumed that at most 70% of any age or risk group would be vaccinated, however there is a natural relationship between vaccine efficacy and proportion vaccinated that generates the same level of overall protection;
- Who is vaccinated: we have consistently shown that prioritising the vaccination of the elderly is by far the most effective strategy for reducing subsequent deaths. It is only for vaccines that have a vastly reduced efficacy in the elderly (below 20%) that other prioritisation orderings become more effective.
In the majority of this work, we have focused on the resultant outbreak after protection by vaccination is complete. At the time of writing all known vaccine candidates require two doses separated by a significant time interval with a further delay between vaccination and protection. If the reproductive ratio remains below one while these events take place, then in general the delays are irrelevant. However, in the scenario where NPIs are relaxed before immunisation of the population is complete, rapid deployment of the vaccine is critical to success (Fig 7).
Our analysis has also focused throughout on two differing outcome measures, both the number of deaths and the expected QALY loss. Given that QALYs give greater weight to preventing illness or death in younger age-groups, it is somewhat surprising that the optimal order of vaccination remains unchanged. This can only be attributed to the far greater severity of disease experienced by the elderly. We have purposefully not performed a full cost-benefit analysis on the vaccine, as in many ways the outcome of this pandemic is difficult to capture in monetary terms. However, under the assumption that R ≈ 1.8, an unvaccinated population would experience a loss of a further 1.58 (1.29-1.78) million QALYs after the first wave of infection, which could be prevented by vaccination of 20.6 (18.9-22.3) million targeted individuals or 41.2 (37.8-44.6) million doses of a vaccine with 90% efficacy (assuming the need for a primary and booster dose). This simple calculation, together with a standard cost-effectiveness threshold of £20,000 per QALY, would suggest that the UK should be willing to pay around £767 (£578-£942) per dose of vaccine—highlighting its huge public health benefit.
Our mathematical model has been carefully matched to the observed dynamics in the UK [14], capturing the fundamental age-structured epidemiology. In contrast, at the time of the original writing of this manuscript in September 2020, there was little information about the characteristics of any new vaccine and the associated model parameters. Therefore, many more details could subsequently be incorporated within the model structure as these became known for any emerging vaccine candidate. Including uptake, efficacy, time between vaccination and becoming immunised, and waning immunity.
For simplicity, we assumed 70% vaccine uptake across all age-groups based on what has been obtainable for vaccines targeting other infections, such as within elder age groups and healthcare workers for the UK seasonal influenza vaccination programme [46]. In practice, vaccine uptake is likely to be regionally and socio-demographically correlated [47, 48]. Such correlations may potentially lead to pockets of high susceptibility in the population which can act as locations of small-scale outbreaks and reduce herd immunity [49]. It is also likely that vaccine uptake will vary in time as the population’s perceived risk varies [50, 51], with high levels of hospitalisations and deaths leading to a greater demand for the vaccine. We expect there to be a complex four-way interaction between levels of infection, NPI policy, NPI adherence and vaccine uptake. From a public health perspective, it is therefore key to understand the drivers of vaccine uptake and vaccine hesitancy, identify groups that may have lower than average uptake and plan accordingly.
Similarly, the assumed efficacy has either been taken as constant across all age-groups or obeying a simple declining function with age; incorporating more realistic estimates about the action of any vaccine is a vital first step in assessing the benefits. Post the initial composition of this article, multiple vaccine manufacturers have peer-reviewed publications presenting the findings of their phase 3 trials. These have generally demonstrated high vaccine efficacy against symptoms [52–54], up to 95% given a two dose schedule for the Pfizer/BioNTech vaccine [52], though as of January 2021 the transmission blocking potential of these vaccines remains unclear. From our set of vaccination characteristic scenarios, the Type 2 vaccine with 90% efficacy coincides most favourably with the attributes discerned from clinical trials thus far.
To a large extent, we have also ignored the time delays between the start of any vaccine program and the protection of the vaccinated individuals; the full implications of any delays can only be calculated once more is known about the mechanisms and limits on delivery, as well as a better understanding of the reproductive number (R) during and after the campaign. Including these elements within the model is feasible, but is presently hampered by a lack of data for parameter inference.
Over longer time scales, the possibilities of waning immunity and mutation may perturb these predictions. Waning immunity, of immunity either naturally derived immunity or immunity induced from vaccination, may necessitate regular vaccination programmes against SARS-CoV-2 protecting the most vulnerable in a similar manner to seasonal influenza vaccines [55]. Currently there is a lack of fundamental understanding of SARS-CoV-2 epidemiology, in particular whether subsequent infections have the same severity as primary infections, as well as the duration of protection. Both of these elements can be factored into the prediction mechanisms, but without detailed evidence such long-term forecasts are speculation.
Finally, the underlying transmission model described is necessarily a simplified representation of reality based on several assumptions. Use of alternative assumptions may have an impact on the determined vaccine priority order. For example, the data informing contact structure for the UK was measured historically. Were contact patterns in early 2020 (pre-lockdown) to substantially differ from the preexisting data, the influence of projected NPIs and vaccination effects may be impacted. Similarly, while we can infer the compliance to the currently imposed rules given a parsimonious approach of an age-independent scaling of the contact arrays, in reality there may be age-specific social distancing effects. However, we had limited understanding of how people would behave as controls are released—would they remain wary of potentially infectious situations, or would they compensate for the time in lockdown. This still remains an open question [56] and is a key policy consideration as restrictions are varied.
This work has focused on the dynamics within the UK, but the robustness of our conclusions, that the vaccine should be optimally targeted at the elderly, suggests that this finding should hold for many countries with similar age-profiles and age-structured mixing patterns. Ultimately, vaccination is likely to be one of the greatest contributing factors in providing a route out of this pandemic, and it is therefore important that the vaccine is deployed as efficiently as possible such that early limited supplies are used to greatest effect.
Supporting information
S1 Text. Description of model formulation, including: The underlying ODE model; an explanation of how social distancing is captured; the public health measurable quantities; key parameters that are inferred through the MCMC scheme; and calculation of QALY (Quality Adjusted Life Year) losses.
https://doi.org/10.1371/journal.pcbi.1008849.s001
(PDF)
References
- 1.
Control for Disease Prevention EC. Covid-19 situation update worldwide, as of 16 august 2020. https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases (accessed August, 2020).
- 2.
NHS UK. Check if you or your child has coronavirus symptoms (2020). https://www.nhs.uk/conditions/coronavirus-covid-19/symptoms/. [Online] (Accessed: 11 December 2020).
- 3.
News B. Coronavirus vaccine: Uk government signs deals for 90 million doses. https://www.bbc.co.uk/news/health-53772650 (accessed August, 2020).
- 4. Graham SP, McLean RK, Spencer AJ, Belij-Rammerstorfer S, Wright D, et al. Evaluation of the immunogenicity of prime-boost vaccination with the replication-deficient viral vectored covid-19 vaccine candidate chadox1 ncov-19. npj Vaccines 5(1):1–6 (2020).
- 5. Folegatti PM, Ewer KJ, Aley PK, Angus B, Becker S, et al. Safety and immunogenicity of the chadox1 ncov-19 vaccine against sars-cov-2: a preliminary report of a phase 1/2, single-blind, randomised controlled trial. The Lancet (2020).
- 6. Bos R, Rutten L, van der Lubbe JE, Bakkers MJ, Hardenberg G, et al. Ad26-vector based covid-19 vaccine encoding a prefusion stabilized sars-cov-2 spike immunogen induces potent humoral and cellular immune responses. bioRxiv (2020). pmid:33083026
- 7. Mulligan MJ, Lyke KE, Kitchin N, Absalon J, Gurtman A, et al. Phase 1/2 study to describe the safety and immunogenicity of a covid-19 rna vaccine candidate (bnt162b1) in adults 18 to 55 years of age: interim report. medRxiv (2020).
- 8.
Valneva. Valneva confirms participation in uk government covid-19 vaccine response program. https://valneva.com/press-release/valneva-confirms-participation-in-uk-government-covid-19-vaccine-response-program/ (accessed August, 2020).
- 9. Tian JH, Patel N, Haupt R, Zhou H, Weston S, et al. Sars-cov-2 spike glycoprotein vaccine candidate nvx-cov2373 elicits immunogenicity in baboons and protection in mice. bioRxiv (2020).
- 10.
GlaxoSmithKline. Gsk vaccine press release. https://www.gsk.com/en-gb/media/press-releases/sanofi-and-gsk-to-join-forces-in-unprecedented-vaccine-collaboration-to-fight-covid-19/ (accessed August, 2020).
- 11. Funk CD, Laferrière C, Ardakani A. A snapshot of the global race for vaccines targeting sars-cov-2 and the covid-19 pandemic. Frontiers in Pharmacology 11:937 (2020). pmid:32636754
- 12. Callaway E. Coronavirus vaccine trials have delivered their first results-but their promise is still unclear. Nature 581 (7809):363–364 (2020). pmid:32433634
- 13. Keeling MJ, Hill E, Gorsich E, Penman B, Guyver-Fletcher G, et al. Predictions of covid-19 dynamics in the uk: short-term forecasting and analysis of potential exit strategies. medRxiv (2020).
- 14. Keeling MJ, Dyson L, Guyver-Fletcher G, Holmes A, Semple MG, et al. Fitting to the UK COVID-19 outbreak, short-term forecasts and estimating the reproductive number. medRxiv page 2020.08.04.20163782 (2020). pmid:32817970
- 15. Li F, Li YY, Liu MJ, Fang LQ, Dean NE, et al. Household transmission of SARS-CoV-2 and risk factors for susceptibility and infectivity in Wuhan: a retrospective observational study. Lancet Infect. Dis. (2021). pmid:33476567
- 16. Pitman R, Nagy L, Sculpher M. Cost-effectiveness of childhood influenza vaccination in england and wales: results from a dynamic transmission model. Vaccine 31(6):927–942 (2013). pmid:23246550
- 17. Baguelin M, Camacho A, Flasche S, Edmunds WJ. Extending the elderly-and risk-group programme of vaccination against seasonal influenza in england and wales: a cost-effectiveness study. BMC medicine 13(1):236 (2015). pmid:26459265
- 18. Datta S, Pink J, Medley GF, Petrou S, Staniszewska S, et al. Assessing the cost-effectiveness of hpv vaccination strategies for adolescent girls and boys in the uk. BMC infectious diseases 19(1):552 (2019). pmid:31234784
- 19. Hill EM, Petrou S, De Lusignan S, Yonova I, Keeling MJ. Seasonal influenza: Modelling approaches to capture immunity propagation. PLoS computational biology 15(10):e1007096 (2019). pmid:31658250
- 20.
Archer SL. Providing care for the 99.9% during the covid-19 pandemic: How ethics, equity, epidemiology, and cost per qaly inform healthcare policy. In Healthcare management forum, page 0840470420939854. SAGE Publications Sage CA: Los Angeles, CA (2020).
- 21. Juneau CE, Pueyo T, Bell M, Gee G, Potvin L. Evidence-based, cost-effective interventions to suppress the covid-19 pandemic: a rapid systematic review. medRxiv (2020).
- 22. Matrajt L, Eaton J, Leung T, Brown ER. Vaccine optimization for covid-19, who to vaccinate first? medRxiv (2020). pmid:32817963
- 23. Bartsch SM, O’Shea KJ, Ferguson MC, Bottazzi ME, Wedlock PT, et al. Vaccine efficacy needed for a covid-19 coronavirus vaccine to prevent or stop an epidemic as the sole intervention. American journal of preventive medicine (2020). pmid:32778354
- 24. Gumel AB, Iboi EA, Ngonghala CN. Will an imperfect vaccine curtail the covid-19 pandemic in the us? medRxiv (2020).
- 25. Mossong J, Hens N, Jit M, Beutels P, Auranen K, et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med 5(3):e74 (2008). pmid:18366252
- 26. Prem K, Cook AR, Jit M. Projecting social contact matrices in 152 countries using contact surveys and demographic data. PLoS computational biology 13(9):e1005697 (2017). pmid:28898249
- 27. Brand SPC, Aziza R, Kombe IK, Agoti CN, Hilton J, et al. Forecasting the scale of the covid-19 epidemic in kenya. medRxiv (2020).
- 28. Barton M, Mehta K, Kumar K, Lu J, le Saux N, et al. COVID- 19 Infection in Children: Estimating Pediatric Morbidity and Mortality. medRxiv preprint pages 1–26 (2020).
- 29. Dong Y, Mo X, Hu Y, Qi X, Jiang F, et al. Epidemiology of COVID-19 Among Children in China. Pediatrics 145(6):e20200702 (2020). pmid:32179660
- 30. Viner RM, Mytton OT, Bonell C, Melendez-Torres G, Ward JL, et al. Susceptibility to and transmission of COVID-19 amongst children and adolescents compared with adults: a systematic review and meta-analysis. medRxiv page 2020.05.20.20108126 (2020).
- 31. Coggon D, Croft P, Cullinan P, Williams A. Assessment of workers personal vulnerability to covid-19 using covid-age. medRxiv (2020). pmid:32761080
- 32. Williamson E, Walker AJ, Bhaskaran KJ, Bacon S, Bates C, et al. Opensafely: factors associated with covid-19-related hospital death in the linked electronic health records of 17 million adult nhs patients. MedRxiv (2020).
- 33.
Royal college of general practitioners (rcgp) research and surveillance centre. https://www.rcgp.org.uk/clinical-and-research/our-programmes/research-and-surveillance-centre.aspx (accessed June, 2020).
- 34.
Audit ND. National diabetes audit mortality analysis 2018. NHS digital (2018).
- 35.
UK CR. Cancer incidence by age. https://www.cancerresearchuk.org/health-professional/cancer-statistics/incidence/age#heading-Zero (accessed June, 2020).
- 36. Davies NG, Kucharski AJ, Eggo RM, Gimma A, Edmunds WJ, et al. Effects of non-pharmaceutical interventions on COVID-19 cases, deaths, and demand for hospital services in the UK: a modelling study. Lancet Public Heal. 5(7):e375–e385 (2020). pmid:32502389
- 37.
Anderson RM, Anderson B, May RM. Infectious diseases of humans: dynamics and control. Oxford university press (1992).
- 38.
Keeling MJ, Rohani P. Modeling infectious diseases in humans and animals. Princeton University Press (2011).
- 39. Sharpe HR, Gilbride C, Allen E, Belij-Rammerstorfer S, Bissett C, et al. The early landscape of covid-19 vaccine development in the uk and rest of the world. Immunology (2020). pmid:32460358
- 40. Lurie N, Saville M, Hatchett R, Halton J. Developing covid-19 vaccines at pandemic speed. New England Journal of Medicine 382(21):1969–1973 (2020). pmid:32227757
- 41. Nguyen LH, Drew DA, Graham MS, Joshi AD, Guo CG, et al. Risk of covid-19 among front-line health-care workers and the general community: a prospective cohort study. The Lancet Public Health (2020). pmid:32745512
- 42.
Digital N. Nhs workforce statistics- feb 2020. https://digital.nhs.uk/data-and-information/publications/statistical/nhs-workforce-statistics/february-2020 (accessed July, 2020).
- 43. Goodwin K, Viboud C, Simonsen L. Antibody response to influenza vaccination in the elderly: a quantitative review. Vaccine (2006). pmid:16213065
- 44. Riley S, Ainslie KE, Eales O, Jeffrey B, Walters CE, et al. Community prevalence of sars-cov-2 virus in england during may 2020: React study. medRxiv (2020).
- 45. Keeling MJ, White PJ. Targeting vaccination against novel infections: risk, age and spatial structure for pandemic influenza in great britain. Journal of the Royal Society Interface 8(58):661–670 (2011). pmid:20943682
- 46.
Public Health England. Official Statistics: Annual flu reports (2020). https://www.gov.uk/government/statistics/annual-flu-reports. [Online] (Accessed: 11 December 2020).
- 47. Bachtiger P, Adamson A, Maclean WA, Quint JK, Peters NS. Inadequate intention to receive Covid-19 vaccination: indicators for public health messaging needed to improve uptake in UK. medRxiv page 2020.12.07.20243881 (2020).
- 48.
Office for National Statistics. Coronavirus and the social impacts on Great Britain: 18 December 2020 (2020). https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/healthandwellbeing/bulletins/coronavirusandthesocialimpactsongreatbritain/18december2020. [Online] (Accessed: 18 December 2020).
- 49. Keeling MJ. The effects of local spatial structure on epidemiological invasions. Proc. R. Soc. Lond. B. 266:859–867 (1999). pmid:10343409
- 50. Bish A, Yardley L, Nicoll A, Michie S. Factors associated with uptake of vaccination against pandemic influenza: A systematic review. Vaccine 29(38):6472–6484 (2011). pmid:21756960
- 51. Sherman SM, Smith LE, Sim J, Amlôt R, Cutts M, et al. COVID-19 vaccination intention in the UK: results from the COVID-19 vaccination acceptability study (CoVAccS), a nationally representative cross-sectional survey. Hum. Vaccin. Immunother. pages 1–10 (2020). pmid:33242386
- 52. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N. Engl. J. Med. 383(27):2603–2615 (2020). pmid:33301246
- 53. Voysey M, Clemens SAC, Madhi SA, Weckx LY, Folegatti PM, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 397(10269):99–111 (2021). pmid:33306989
- 54. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N. Engl. J. Med. page NEJMoa2035389 (2020). pmid:33378609
- 55.
World Health Organisation. Vaccination—WHO/Europe | Influenza (2020). https://www.euro.who.int/en/health-topics/communicable-diseases/influenza/vaccination. [Online] (Accessed: 17 December 2020).
- 56. Bavel JJV, Baicker K, Boggio PS, Capraro V, Cichocka A, et al. Using social and behavioural science to support COVID-19 pandemic response. Nat. Hum. Behav. 4(5):460–471 (2020). pmid:32355299