https://orcid.org/0000-0002-5133-1382
Figures
Abstract
Introduction
Vaccination programs aim to control the COVID-19 pandemic. However, the relative impacts of vaccine coverage, effectiveness, and capacity in the context of nonpharmaceutical interventions such as mask use and physical distancing on the spread of SARS-CoV-2 are unclear. Our objective was to examine the impact of vaccination on the control of SARS-CoV-2 using our previously developed agent-based simulation model.
Methods
We applied our agent-based model to replicate COVID-19-related events in 1) Dane County, Wisconsin; 2) Milwaukee metropolitan area, Wisconsin; 3) New York City (NYC). We evaluated the impact of vaccination considering the proportion of the population vaccinated, probability that a vaccinated individual gains immunity, vaccination capacity, and adherence to nonpharmaceutical interventions. We estimated the timing of pandemic control, defined as the date after which only a small number of new cases occur.
Results
The timing of pandemic control depends highly on vaccination coverage, effectiveness, and adherence to nonpharmaceutical interventions. In Dane County and Milwaukee, if 50% of the population is vaccinated with a daily vaccination capacity of 0.25% of the population, vaccine effectiveness of 90%, and the adherence to nonpharmaceutical interventions is 60%, controlled spread could be achieved by June 2021 versus October 2021 in Dane County and November 2021 in Milwaukee without vaccine.
Citation: Alagoz O, Sethi AK, Patterson BW, Churpek M, Alhanaee G, Scaria E, et al. (2021) The impact of vaccination to control COVID-19 burden in the United States: A simulation modeling approach. PLoS ONE 16(7): e0254456. https://doi.org/10.1371/journal.pone.0254456
Editor: Martin Chtolongo Simuunza, University of Zambia, ZAMBIA
Received: March 29, 2021; Accepted: June 27, 2021; Published: July 14, 2021
Copyright: © 2021 Alagoz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All underlying data is included in the paper.
Funding: This work was supported by the National Institutes of Health under National Institute of Allergy and Infectious Diseases (NIAID) Grant 1DP2AI144244-01. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: Oguzhan Alagoz is a paid consultant for Bristol Myers Squibb and Biovector Inc, outside of the submitted work. Other authors have no conflicts of interest to report. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
With over 32 million cases in the US alone, poorly controlled transmission of SARS-CoV-2 has challenged the capacity of health systems and has resulted in over 580,000 deaths [1]. Until recently, the only effective measures for controlling the spread of SARS-CoV-2 have been nonpharmaceutical interventions (NPIs), such as physical distancing and masking. Sustained adherence to these NPIs is variable and difficult to achieve [2].
The US Food and Drug Administration authorized the use of two, two-dose vaccines and one, single-dose vaccine against SARS-CoV-2 of similarly high efficacy to prevent symptoms of novel coronavirus disease (COVID-19) and hospitalization [3, 4]. As of May 12, 2021, 36% of the US population has been fully vaccinated [1]. However, if and when widespread use of these vaccines will result in sustained control of the pandemic remains unclear given the speed of vaccine roll out and societal factors such vaccine hesitancy and suboptimal adherence to NPIs. Moreover, there is a need to examine these factors at a regional level given the rolling nature of the pandemic and COVID-19 hot spots around the country.
The objective of this study was to use our previously developed COVID-19 agent-based simulation model [2], to examine the impact of vaccine coverage and effectiveness, vaccination capacity, and adherence to NPIs on SARS-CoV-2 burden, as well as to predict how these factors influence control of virus spread in urban communities in the US.
Methods
This study used only publicly available de-identified data. We previously developed the COVID-19 Agent-based simulation Model (COVAM) [2] to represent the interactions among people that may lead to transmission of SARS-CoV-2 in three urban regions in the US: Dane County in Wisconsin, the Milwaukee metro area in Wisconsin, and New York City (NYC). Briefly, COVAM works as follows: individuals who belong to one of eight possible states representing an individual’s COVID-19-related status (S1 Fig in S1 Data) interact with each other through which SARS-CoV-2 is transmitted. The number of such close interactions in a given day is estimated separately for different regions considering population densities using data from social network literature and calibration [5, 6]. For each of these daily interactions, there is a possibility that a contagious individual exposes another individual to SARS-CoV-2. The number of daily contacts differs by age group, with older individuals less likely to have such interactions as compared to younger individuals. COVAM considers the possibility that not all individuals infected with SARS-CoV-2 will be tested positive and reported. Moreover, COVAM considers the possibility that some asymptomatic individuals transmit the disease prior to showing symptoms. COVAM also assumes that individuals who are experienced a previous COVID-19 infection gain protective immunity for future infections. The basic reproduction number (R0) corresponding to the unmitigated base-case transmission dynamics was 3.34 for Dane County and Milwaukee and 6.68 for NYC due to a large number of daily contacts.
A unique feature of COVAM is its ability to represent varying levels of adherence of the local communities to NPIs implemented in different regions. COVAM uses cell-phone mobility data and calibration to estimate a time-dependent adherence to NPIs and implements it explicitly by adjusting the number of contacts per person each day [7–9]. For instance, a 70% adherence level to NPIs in Dane County and Milwaukee is implemented by simply reducing the number of daily contacts from 10 per day to 3 per day per person, which slows the transmission of SARS-CoV-2. COVAM uses adherence to NPIs to model several distinct behaviors to mitigate transmission, including mask wearing and reduced number of interactions where individuals do not maintain a 6-feet distance during person-to-person interactions, and estimate future burden of COVID-19. COVAM has the ability to represent age-specific adherence to NPIs. Similarly, COVAM allows individuals to show dynamic behavior and have varying levels of adherence to NPIs over time.
COVAM was calibrated using historical pandemic data from the three urban regions and validated in the short term with data that were not used in model development. COVAM is programmed using C++ programming language for flexibility and quicker computational times. All experiments in this study are conducted in personal computers. Additional details are available in Alagoz et al. [2] and in the Supplement.
COVAM updates
We made two extensions to COVAM to answer the research questions of this study. The original version of COVAM used a fixed level of adherence to NPIs for future projections. However, cell-phone-based data and several studies have shown that the response of local communities to NPIs changed over time; therefore, the assumption of a fixed adherence level may not be realistic [7–10]. Specifically, even in the absence of seasonal changes, the observed number of cases and deaths impacted how local communities followed NPIs [10–12]. To this end, we added a dynamic adherence scenario in which we defined two thresholds that trigger high or low adherence to NPIs. Namely, if the number of daily new cases exceeds a high threshold, then the NPI adherence level increases significantly from observed levels. Similarly, if the number of daily new cases drops below a low threshold, then the adherence level drops significantly.
The second extension was the incorporation of vaccination into COVAM. We assumed that some individuals undergo vaccination and as a result may gain protective immunity and become non-susceptible after vaccination representing the vaccine effectiveness rate. We assumed that individuals who gain protective immunity after vaccination cannot get infected with COVID-19 and do not transmit the disease to others. We also considered that only a proportion of the population agrees to be vaccinated, and there is a daily capacity for vaccination as a function of the proportion of the population in the community. We also allowed vaccinated individuals (whether they are immunized or not) to have a lower adherence to NPIs. We did not explicitly model the timeline between first and second doses for the two-dose vaccines. Instead, we assumed that immunization is developed 14 days after the second dose of the vaccine is administered for the two-dose vaccines.
Finally, we calibrated COVAM to the latest data on the reported number of confirmed cases in Dane County, Milwaukee, and NYC using case counts by January 5, 2021, January 5, 2021, and January 14, 2021, respectively and compared COVAM’s predictions against data until February 1, 2021.
Vaccination scenarios
In all runs, we assumed that vaccination started on January 5, 2021, as the first vaccine was administered on December 14, 2020 in the US [13]. We used COVAM to evaluate the impact of different aspects of vaccination and NPI adherence using the model inputs as presented in Table 1. We used observed adherence levels until February 1, 2021 and assumed that adherence levels after this date are maintained afterwards. The adherence scenarios in Table 1 are based on the new cases and the adherence levels observed in these regions since the beginning of the pandemic (S1 Table in S1 Data).
We ran 100 replications for each experiment to obtain stable estimates and report only mean values due to very low standard errors. Unless noted otherwise, we reported the cumulative number of confirmed cases associated with each scenario on December 31, 2021.
Our base case scenario assumed that vaccination starts on January 5, 2021; daily vaccination capacity is 0.25% of the population per day (i.e., 1,350 people/day in Dane County, 4,065 people/day in Milwaukee, and 21,680 people/day in NYC); there is a 20% drop in adherence among vaccinated individuals; vaccine effectiveness is 90%; vaccination coverage is 50%; and the baseline test rate is 75%.
Controllable spread date analysis
Using the implied R0 values for infectious disease models, the herd immunity is theoretically achieved when 70%, 70%, and 85% of the population gains immunization through recovery from COVID-19 or vaccination in Dane County, Milwaukee, and NYC, respectively. However, these herd immunity levels assume that NPIs, including face mask use, are no longer adopted. Therefore, we investigated whether vaccination, along with NPIs, will mitigate the pandemic to a level such that that it will become controllable. For this purpose, we defined a controllable spread date as the date after which the number of daily new confirmed cases never exceeds 20 for Dane County, 60 for Milwaukee, and 320 for NYC. For example, if the daily number of new cases never exceeds 20 for Dane County after June 15, 2021, the controllable spread date is set to June 15, 2021. We selected these daily numbers of confirmed cases based on data from Dane County that employs 180 contact tracers to trace COVID-19 cases; therefore, 20 new cases per day would make it feasible to conduct aggressive and efficient contact tracing to completely control the disease in Dane County [14]. For the other regions, we scaled up the number considering population size.
Sensitivity analysis
We conducted a parametric sensitivity analysis in which we tested the impact of uncertainty in two input parameters: drop in adherence to NPIs after vaccination, where the drop in adherence was assumed to be 0% as opposed to 20% in the base case, and baseline test rate, where the probability of testing in earlier days of the pandemic was assumed to be 50% as opposed to 75% in the base case. We also conducted a structural sensitivity analysis in which we assumed that vaccination reduces the risk of infection instead of leading to complete immunity. That is, a 90% vaccine effectiveness scenario reduces the risk of getting infected among all vaccinated individuals by 90% but does not lead to complete immunity.
Results
COVAM accurately predicted the reported number of cases in each urban area in the short term (S2 Fig in S1 Data). We first reported the number of confirmed cases over time for different vaccine coverage and adherence to NPI scenarios when vaccine effectiveness is 90% and daily vaccination capacity is 0.25% (Fig 1). We found that the total number of confirmed cases was not sensitive to the vaccination coverage in any of the regions as long as communities keep a high level of adherence to NPIs. In general, the dynamic adherence scenario has led to later controllable spread dates compared to those under fixed adherence scenarios for both vaccination and no vaccination cases.
Fig 2 and Table 2 show how adherence to NPIs change the effect of vaccination on the cumulative number of cases. Assuming vaccine effectiveness is 90%, vaccine coverage is 50%, and daily vaccination capacity is 0.25%, we found that the level of adherence to NPIs had a major impact on the cumulative number of cases (Fig 2 and Table 2), as well as the controllable spread date. When the level of NPI adherence is very high (75% for Dane County and Milwaukee; 90% for NYC), the effect of vaccination on the controllable spread date and number of cumulative cases was minimal compared to no vaccination. When the NPI adherence rates are equal to 70% for Dane County and Milwaukee, and 85% for NYC, vaccination reduced the number of confirmed cumulative cases from 51,784, 185,343, and 1,110,070 to 43,326, 165,679, and 784,137implying an 16%, 11%, and 29% rate of reduction in the number of confirmed cases for Dane, Milwaukee, and NYC, respectively, and reduced the time to controllable spread by 2–3 months in all three regions (Table 2, 0.25% vaccination capacity).
The reduction in the number of cases due to vaccination was higher in Dane County compared to Milwaukee. The proportion of cumulative confirmed cases in the population in Dane County and Milwaukee as of January 31, 2021 were 7% and 10%, respectively [15]. COVAM considers the possibility that the actual number of cases is higher than the confirmed number of cases due to some patients experiencing very mild disease and the limited testing. As such, COVAM estimated that the proportion of cumulative confirmed and unreported cases in the population in Dane County and Milwaukee as of January 31, 2021 was 8% and 11%, respectively. Therefore, the prevalence of SARS-CoV-2 was higher in Milwaukee compared to Dane County during the vaccination period.
Table 2 also shows that maintaining a high level of adherence to NPIs under the same vaccination scenario reduces the number of confirmed cases and results in a significantly earlier date of controllable spread compared to lower NPI adherence in all three regions. For example, in Dane County, a 60% level of NPI adherence after February 1, 2021 led to 53,232 cases with vaccination and a controllable spread date of June 15, 2021 versus a 70% level of NPI adherence, which led to a total of 43,326 cases and a controllable spread date of March 29, 2021.
We also found that daily vaccination capacity had a differential effect on the number of cases and the date on which controllable spread is achieved (Table 2). In particular, under the dynamic adherence scenario, vaccination at 0.5% instead of 0.05% of the population per day reduced the number of confirmed cases from 72,062, 250,540, and 1,273,560 to 42,468, 163,134, and 688,177 implying a 41%, 35%, and 46% reduction in Dane County, Milwaukee, and NYC, respectively, and reduced the time to controllable spread by 12, 10, and 16 months, respectively.
Table 3 shows that the effectiveness of vaccination does not have a major impact on the date when controllable spread is achieved, whereas it reduces the number of confirmed cases greatly. Parametric and structural sensitivity analyses recapitulated the overall trends observed in the base-case runs (S3-S10 Figs in S1 Data and S3-S8 Tables in S1 Data).
Discussion
In this simulation study, we estimated the impact of vaccination on the number of COVID-19 cases in three urban communities using agent-based simulation modeling. We found that controllable spread of SARS-CoV-2 can be achieved sooner than when a large proportion of the population is vaccinated (e.g., 70–80%) as long as there is high adherence to NPIs in the community. We further found that vaccination would reduce the number of COVID-19 cases significantly, but the rate of reduction in the number of cases differs among regions. Finally, we found that the level of adherence to NPIs has a major impact on the number of cases, as well as the date that controllable spread is achieved regardless of vaccine capacity, vaccine effectiveness, or region. In NYC, vaccination would reduce the number of cases from 3,307,090to 1,379,770, or by 58%, when the adherence to NPIs is fixed at 80%. Under the same scenario, controllable spread would be achieved in June 2021 as opposed to September 2021 without vaccination.
Although the disease spread rate (i.e., R0 value) was identical for Dane County and Milwaukee, the impact of vaccination on the number of cases and timing of controllable spread was more pronounced in Dane County compared to Milwaukee. Of note, according to the COVAM model, 8% of the Dane County population was infected as of January 31, 2021 compared to 11% of the Milwaukee population. Thus, our results imply that vaccination may have a larger impact when the history of COVID-19 infections is lower to start with. Based on this, we suggest to benefit most from vaccination, regions need to keep a high level of adherence to NPIs to keep caseload low, a similar finding as reported in a prior modeling study [16]. Relying on vaccination alone to control the spread of COVID-19 over the coming months may delay the timing of controllable disease by several months, and this has important implications for the timing of lifting policies for NPIs that are currently in place.
Mainstream media, as well as prior modeling efforts, have focused on the endpoint of herd immunity–the point at which disease transmission is halted and life can “return to normal” from the standpoint of NPIs [17, 18]. While reaching herd immunity is the ultimate goal of a vaccination campaign, we chose to focus on an intermediate metric: controlled spread while continuing reasonable public health precautions. Our results highlight the importance of continued adherence to NPIs during vaccine rollout. As the vaccine is being delivered, our results suggest that vaccine effectiveness in reducing viral spread is highly dependent on NPIs. If NPI efforts are sustained at current levels during the rollout, then even low levels of vaccine uptake will be able to control viral spread. On the other hand, if vaccine rollout is paired with NPI easing, a higher proportion of the population will need vaccination to stop the accumulation of new cases.
Our findings are consistent with those from a few other modeling studies that report reaching herd immunity depends on high adherence to NPIs during vaccination rollout [16, 19]. The impact of even highly effective vaccines may be diminished if deployed into a population with high viral transmission caused by low adherence to NPIs [16]. In addition, the use of NPIs may decrease the level of vaccine coverage needed to achieve herd immunity [19]. Our study extends the existing literature by showing that the degree to which NPIs, vaccine coverage, and pace of vaccination impact disease control may vary regionally, and are highly dependent on both population density and the existing cumulative burden of COVID-19 since the pandemic began. Areas of lower density need lower adherence to NPIs to decrease case rates with vaccination, and those with a lower existing burden of COVID-19 have a greater ability to improve their trajectories with vaccination campaigns.
Our study’s findings are also consistent with a few other agent-based simulation models that examined the impact of vaccination on pandemic control [20, 21]. One recent study focused on the pandemic control in the Ontario region in Canada and found that relaxing NPIs and lockdown restrictions early can lead to delayed controllable spread or a second wave of infections [20]. Similarly, we found that dynamic adherence to NPIs was associated with lower infection reduction compared to steady rates of NPI adherence. All vaccination experiments shown in Fig 2 illustrate an additional wave of infections under dynamic adherence to NPIs. Our study also agrees with another study that focused on the US and reported a modest impact of vaccine effectiveness in some cases [21]. Similarly, as shown in Table 3, we also found that greater vaccine effectiveness was not strongly associated with greater reduction in infections.
Our study has several limitations related to uncertainty in vaccine effectiveness in real-world settings. Authorized COVID-19 vaccines to date have demonstrated high efficacy for preventing COVID-19 illness and hospitalization [3]. COVAM assumes that vaccination also prevents transmission of SARS-CoV-2. However, studies to determine the effect of vaccination on acquisition and shedding of SARS-CoV-2 are ongoing. Our model also assumes that COVID-19 vaccines will be effective in preventing transmission of new and future variants of the virus. The B.1.1.7 (UK), B.1.351 (South Africa), and P.1 (Brazil) variants have all been detected in the US [22], and all contain mutations in the spike protein. Both the Pfizer and Moderna vaccines are believed to be effective against the B.1.1.7 variant [23–25]. However, a recent study suggests the neutralizing antibodies induced by vaccination are less potent against the B.1.351 and B.1.1.7 variants in vitro [26]. As long as SARS-CoV-2 is replicating at high levels, new variants can be expected to emerge, underscoring the importance of high-level adherence to NPIs and rapid vaccination with high-level coverage. Furthermore, we do not use a full calibration procedure that is commonly used in simulation modeling to estimate the unobservable inputs of the model, which may have led to suboptimal set of inputs [27–29]. Finally, our model does not consider age-based differences in the administration and effectiveness of the vaccines. However, currently used vaccines are administered for individuals over ages 16 only. It is unknown if vaccination of individuals younger than 16 of age will begin by the end of the study’s simulation period or if the effectiveness will be different for younger individuals.
In conclusion, this simulation modeling demonstrates that continued high adherence to NPIs along with vaccination results in a shorter time to control the COVID-19 pandemic in US urban areas. Furthermore, our results suggest adhering to NPIs to keep the caseload low until vaccine becomes widely available can lead to greater benefit of vaccination in terms of reduction in the number of COVID-19 cases.
References
- 1.
Johns Hopkins University & Medicine. Coronavirus Resource Center 2021 [Available from: https://coronavirus.jhu.edu/.
- 2. Alagoz O, Sethi AK, Patterson BW, Churpek M, Safdar N. Effect of Timing of and Adherence to Social Distancing Measures on COVID-19 Burden in the United States: A Simulation Modeling Approach. Annals of internal medicine. 2020.
- 3. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. New England Journal of Medicine. 2020. pmid:33378609
- 4.
FDA. COVID-19 Vaccines: Emergency Use Authorization—Vaccines 2021 [Available from: https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/covid-19-vaccines#eua-vaccines.
- 5. Del Valle SY, Hyman JM, Hethcote HW, Eubank SG. Mixing patterns between age groups in social networks. Social Networks. 2007;29(4):539–54.
- 6. Lee BY, Brown ST, Cooley P, Potter MA, Wheaton WD, Voorhees RE, et al. Simulating school closure strategies to mitigate an influenza epidemic. Journal of Public Health Management and Practice. 2010;16(3):252–61. pmid:20035236
- 7.
GeoDS Lab @ UW-Madison. Mapping Mobility Changes in Response to COVID-19 2021 [Available from: https://geods.geography.wisc.edu/covid19/physical-distancing/.
- 8.
Google. COVID-19 Community Mobility Report 2021 [Available from: https://www.google.com/covid19/mobility/.
- 9.
Unacast. Social Distancing Scoreboard 2021 [Available from: https://www.unacast.com/covid19/social-distancing-scoreboard.
- 10. Masters NB, Shih S-F, Bukoff A, Akel KB, Kobayashi LC, Miller AL, et al. Social distancing in response to the novel coronavirus (COVID-19) in the United States. PloS one. 2020;15(9):e0239025. pmid:32915884
- 11. Pedersen MJ, Favero N. Social Distancing during the COVID‐19 Pandemic: Who Are the Present and Future Noncompliers? Public Administration Review. 2020;80(5):805–14.
- 12. Xie K, Liang B, Dulebenets MA, Mei Y. The impact of risk perception on social distancing during the COVID-19 pandemic in China. International journal of environmental research and public health. 2020;17(17):6256. pmid:32867381
- 13.
BBC. Covid-19: First vaccine given in US as roll-out begins 2021 [Available from: https://www.bbc.com/news/world-us-canada-55305720.
- 14.
PBS Wisconsin. Dane County Shifts to Contact Tracing ’Crisis Model’ Amid Another Record-Setting Day 2021 [Available from: https://pbswisconsin.org/news-item/dane-county-shifts-to-contact-tracing-crisis-model-amid-another-record-setting-day/.
- 15.
Wisconsin Department of Health Services. COVID-19: Wisconsin Data 2021 [Available from: https://www.dhs.wisconsin.gov/covid-19/data.htm.
- 16. Paltiel AD, Schwartz JL, Zheng A, Walensky RP. Clinical Outcomes Of A COVID-19 Vaccine: Implementation Over Efficacy: Study examines how definitions and thresholds of vaccine efficacy, coupled with different levels of implementation effectiveness and background epidemic severity, translate into outcomes. Health Affairs. 2021:
- 17. Randolph HE, Barreiro LB. Herd immunity: understanding COVID-19. Immunity. 2020;52(5):737–41. pmid:32433946
- 18.
McNeil Jr. DG. How Much Herd Immunity Is Enough? New York Times. December 24, 2020.
- 19. Iboi EA, Ngonghala CN, Gumel AB. Will an imperfect vaccine curtail the COVID-19 pandemic in the US? Infectious Disease Modelling. 2020;5:510–24. pmid:32835142
- 20. Vilches TN, Zhang K, Van Exan R, Langley JM, Moghadas SM. Projecting the impact of a two-dose COVID-19 vaccination campaign in Ontario, Canada. Vaccine. 2021;39(17):2360–5. pmid:33812742
- 21. Moghadas SM, Vilches TN, Zhang K, Wells CR, Shoukat A, Singer BH, et al. The Impact of Vaccination on Coronavirus Disease 2019 (COVID-19) Outbreaks in the United States. Clinical Infectious Diseases. 2021. pmid:33515252
- 22.
CDC. Emerging SARS-CoV-2 Variants 2021 [Available from: https://www.cdc.gov/coronavirus/2019-ncov/more/science-and-research/scientific-brief-emerging-variants.html.
- 23. Wu K, Werner AP, Moliva JI, Koch M, Choi A, Stewart-Jones GB, et al. mRNA-1273 vaccine induces neutralizing antibodies against spike mutants from global SARS-CoV-2 variants. BioRxiv. 2021 pmid:33501442
- 24. Greaney AJ, Loes AN, Crawford KH, Starr TN, Malone KD, Chu HY, et al. Comprehensive mapping of mutations to the SARS-CoV-2 receptor-binding domain that affect recognition by polyclonal human serum antibodies. bioRxiv. 2021:https://doi.org/10.1101/2020.12.31.425021.
- 25. Weisblum Y, Schmidt F, Zhang F, DaSilva J, Poston D, Lorenzi JC, et al. Escape from neutralizing antibodies by SARS-CoV-2 spike protein variants. Elife. 2020;9:e61312. pmid:33112236
- 26. Wang P, Lihong L, Iketani S, Luo Y, Guo Y, Wang M, et al. Increased Resistance of SARS-CoV-2 Variants B. 1.351 and B. 1.1. 7 to Antibody Neutralization. bioRxiv. 2021:https://doi.org/10.1101/2021.01.25.428137.
- 27. Barker AK, Alagoz O, Safdar N. Interventions to reduce the incidence of hospital-onset Clostridium difficile infection: An agent-based modeling approach to evaluate clinical effectiveness in adult acute care hospitals. Clinical Infectious Diseases. 2018;66(8):1192–203. pmid:29112710
- 28. Cevik M, Ergun MA, Stout NK, Trentham-Dietz A, Craven M, Alagoz O. Using active learning for speeding up calibration in simulation models. Medical Decision Making. 2016;36(5):581–93. pmid:26471190
- 29. Codella J, Safdar N, Heffernan R, Alagoz O. An agent-based simulation model for C. difficile infection control. Medical Decision Making. 2015;35(2):211–29. pmid:25112595