OpinionAbstract
Compiling the best available scientific evidence to date, we recently demonstrated that natural immunity to COVID-19 is at least as robust as vaccine immunity and that the benefit of vaccinating previously infected and recovered individuals is so modest that it would not likely outweigh vaccine-associated risks. Almost immediately following the peer-reviewed publication of our paper, in what appeared to be a hurried reaction, the CDC released in their MMWR an unusually low-quality scientific paper in a seemingly desperate attempt to defend their dogmatic, one-size-fits-all position on COVID-19 vaccines. In this letter, we summarize the findings of our systematic review, critically review the CDC article, and shed light on the lack of statistical validity and scientific merit of the widely publicized CDC study. As things stand, any recommendation to vaccinate previously COVID-19-infected and recovered individuals, let alone mandate, and force such vaccinations on them, remains scientifically unfounded and potentially harmful.
Keywords: COVID-19; Natural Immunity; Public Health; SARS-CoV-2; Vaccine
Abbreviations: CDC: Control and Prevention; RR: Relative Risk; NNT: Needed To Treat; AR: Absolute Risk
Introduction
Validity is the ability of a research tool or experiment to accurately measure a predefined endpoint in order to assess a rational hypothesis. But, when erroneous conclusions are drawn based on a convoluted study design that lacks validity, this becomes bad science. Worse yet, when the scientific process is undertaken teleologically with a predetermined outcome as its goal, it becomes misconduct. If teleological “science” is intentionally generated to help promote a politically or financially biased narrative, serious harm to individuals and society at large may ensue. In line with those definitions, the paper recently published by the Centers for Disease Control and Prevention (CDC) in their Morbidity and Mortality Weekly Report, which claims superiority of vaccine immunity over natural immunity, represents the textbook definition of poor validity and, at the very least, bad science [1].
Background: A Teleological Emergency?
Through a systematic review and pooled analysis of the literature, we recently compiled the best available scientific evidence comparing the effectiveness of natural immunity and vaccine immunity to COVID-19 [2]. Using stringent inclusion criteria, we limited our analysis to 7 high-quality publications, including a total of 279,107 patients and 56,161 patient-years of follow-up. Compared with the unvaccinated COVID-19-naive cohort, vaccinated patients (i.e. the “vaccine immunity cohort”) had a significantly reduced infection rate (0.9% vs. 4.7% per personyear), yielding a number needed to treat (NNT) of only 6.5 patients to prevent 1 COVID-19 infection per year. Similarly, the previously infected, unvaccinated cohort (i.e., the “natural immunity cohort) had a significantly reduced risk of reinfection (0.25% per personyear) compared with the same COVID-19- naive cohort. When compared head-to-head with the natural immunity group, the vaccine immunity cohort had a 1.86 relative risk (RR) of infection, which was found to be non-statistically significant, but a 0.049% absolute risk (AR) increase, which was statistically significant. Thus, natural immunity against COVID-19 was found to be at least equivalent to vaccine immunity in conferring protection against infection or reinfection. Among all groups, the risk of COVID-19 infection/reinfection was lowest (0.15% per person-year) in previously infected, vaccinated individuals, suggesting a marginal but statistically significant incremental benefit of vaccination in the previously infected and recovered population (RR: 1.82, AR reduction: 0.0039%).
This small benefit of vaccination translated into a very high NNT of 218 in the previously infected and recovered cohort, raising serious doubts about the favorability of the risk-benefit ratio of routine vaccination in this population, even if only well-established, short-term risks of vaccination are taken into account (potential long-term risks, especially in young people, remain unknown at this point). Almost immediately following the publication of our study after in-depth peer review, the CDC released a paper in their Morbidity and Mortality Weekly Report (MMWR), reporting on a cross-sectional study of hospitalized patients with “COVID-19- like illness” within a network of several US hospitals (the “VISION Network”), which putatively demonstrates the superiority of vaccine immunity over natural immunity, thereby attempting to negate the findings of our study [1]. Not surprisingly, the CDC paper was immediately and heavily publicized by media companies and on social media, claiming that vaccines are five times as effective as natural immunity in this regard and that the debate on this issue has essentially been settled in favor of vaccines [3-9]. The implications of this study, according to CDC, is that “all eligible persons should be vaccinated against COVID-19 as soon as possible, including unvaccinated persons previously infected with SARS-CoV-2” [1]. Unfortunately, as stated above and detailed below, the CDC paper represents bad science, of the kind we warn our youngest and most inexperienced research students to avoid at all costs, as they start learning the basics of medical and clinical research. Political analysis and opinions remain beyond the scope of this letter. Specifically, the reasons and exact circumstances underlying this scientific mishap by the CDC, traditionally a highly respected and credible source of medical information, will not be discussed here. Only the merits (lack thereof) of their paper will be discussed.
The Fundamental Problem: Study Validity (Lack Thereof)
The fundamental and primordial problem with the CDC study
is its total lack of validity. The CDC sought to compare protection
against COVID-19 infection/reinfection between vaccinated
patients and those unvaccinated, but with natural immunity from
a previous infection. Unfortunately, what they ended up measuring
in this study was a totally unrelated and quite irrelevant endpoint.
Rather than using a longitudinal, population-based (community +
hospital) observational cohort design to help answer this research
question, the authors relied on an awkward hospital-based crosssectional
design, looking at all patients within their VISION Network
hospitals who, in the first 8 months of calendar year 2021 (January
1 - September 2):
1) Were admitted with a COVID-19-like illness (i.e., largely a
population of patients with various flu syndromes),
2) Underwent molecular testing for COVID-19, and
3) Had, 3-6 months earlier, either had a laboratory-proven
COVID-19 infection or completed a two-dose vaccination with
an FDA-approved mRNA vaccine.
Interestingly, of an initial grand total of 201,269 hospitalizations, only 7,348 patients (3.7% of the entire cohort), 6,328 in the vaccine immunity group and 1,020 in the natural immunity group, satisfied the inclusion criteria and were analyzed. The authors compared the proportions of COVID-19 positive tests in the two groups of patients and found a higher rate of COVID-19 positivity in the unvaccinated, previously infected group, with a crude odds ratio of 1.77 (8.7% vs. 5.1%). Using ill-defined, seemingly acrobatic, and largely opaque statistical adjustments and propensity-based calculations (not detailed or explained in the paper), the authors present a final adjusted odds ratio of 5.49 in favor of vaccines. Those and other methodological red flags and flaws will be discussed below. However, the most fundamental flaw of this paper, its absolute lack of validity, needs to be addressed first. In fact, what the authors claim they have proven is not at all what their data has actually shown. At best, the authors can conclude that a hospitalized patient with clinical symptoms suspicious for COVID-19 who [3-6] months ago, had prior infection with SARS-CoV-2, would be more likely (1.8- 5.5x) to test positive for COVID- 19, but less likely to test positive for other flu viruses or respiratory infections than an otherwise similar patient that, 3-6 months ago, received an mRNA vaccine. In other words, what the authors measure here is merely the rate of COVID-19 positivity relative to other infectious agents with similar clinical presentation in each of those two patient populations. Given the lack of longitudinal follow-up, this does not at all mean that vaccinated patients developed less COVID-19 infections than their naturally immune counterparts. For instance, one could potentially argue that mRNA vaccines might have led to higher rates of viral illnesses and hospitalizations relative to natural immunity, but that its negative impact on non-COVID-19 infections might have been even worse than that on COVID-19, hence a lower proportion of in-hospital SARS-CoV-2 positivity relative to other infectious agents. If that assumption was true, then increased rates of both COVID-19 and COVID-19-like illnesses and related hospitalizations would be accurately uncovered by a longitudinal observational cohort study. In contrast, a cross-sectional study design would lead to the erroneous conclusion that the relative rate of COVID-19 is lower in the vaccinated group. Interestingly enough, in the CDCanalyzed in-hospital cohort, the absolute number of patients with COVID-19-like illnesses who were previously vaccinated is over 6 times larger than that of patients with natural immunity (6,328 vs. 1,020). Even the absolute number of hospitalized patients with laboratory-proven COVID-19 is over 3.5 times higher in the vaccinated group (324 vs. 89). Such ratios (6:1 and 3.5:1) are way out of proportion to the rates of vaccination in the US population. Perhaps, one would rather conclude from this study that, in sharp contrast to the authors’ claim, patients with natural immunity tend to stay healthier and away from hospitals compared with those who received mRNA vaccines.
More Methodological Frailty: Flaws, Biases, and Red Flags
Aside from the fundamental validity problem presented above, the CDC study is replete with methodological mishaps, which we summarize below.
Selection Bias: The analyzed patient cohort is relatively very small, representing only 3.7% of the original cohort of hospitalized patients with COVID-19-like illness. While large numbers of patients had to be excluded based on the authors’ (otherwise reasonable) inclusion criteria, such a large number of excluded patients almost invariably introduces significant selection biases into the statistical analysis.
Subject Misclassification: To be included in the vaccine immunity cohort, a patient had to have had at least 1 negative molecular COVID-19 test, at least 14 days prior to the index hospitalization. Given that a single negative test does not cover the entire 3 to 6-month period preceding the index hospitalization, it is entirely possible that many patients with prior undiagnosed COVID-19 infections were mistakenly misclassified into the vaccine immunity group, which could potentially affect the results of the study.
Unorthodox Statistical Adjustments: The questionable, poorly defined, very opaque statistical adjustments and propensitybased calculations performed by the authors managed to convert a crude odds ratio of only 1.77 into an “adjusted” odds ratio of 5.5 in favor of vaccines, which has since been widely publicized by the media (i.e. vaccines are being advertised as “five times more effective” than natural immunity) [3-9]. For instance, propensity score matching should generally not be used in groups with very little overlap, since it can introduce significant error. Yet, their data set falls precisely under this category. For the sake of transparency and credibility, we invite the authors to publish a follow-up publication detailing their statistical methodology and presenting their raw data.
Selective Time Filtering: The authors excluded patients with prior COVID-19 infection 14-90 days before the index hospitalization and those with mRNA vaccinations over 6 months prior. This convenient cherry-picking is likely to favor vaccine immunity by design, given that natural immunity is typically robust in the weeks following COVID-19 infection, while vaccine immunity has been shown to wane after 6 months [10-14].
Exclusion of Janssen Vaccine Recipients: It is a blaring fact that the authors entirely excluded recipients of the Janssen vaccine, which is generally known to be the least effective of the 3 FDA-approved COVID-19 vaccines. Excluding this small subset of vaccinated subjects from the analysis is an undeniable “fudge factor” in this paper, which is likely to skew the analysis even further in favor of vaccine immunity.
Conclusion
The paper recently published by the CDC has little, if any,
merits and should have no impact whatsoever on the decision to
vaccinate previously COVID-19-infected and recovered individuals,
let alone mandate and force such vaccinations on them [1]. We urge
the CDC to retract this paper or, at the very least, issue a clear and
unequivocal statement acknowledging its severe methodological
limitations. This agency’s credibility is truly at stake here. The
CDC has to strive to free itself from overreaching politics, regain
the trust of the population, and restore its image as a uniquely credible and authoritative public health resource for America and
the world. Finally, we would like to reiterate that, to date, the best
available evidence on the topic of natural vs. vaccine COVID-19
immunity is clearly laid out in our recent systematic review [2], and
its conclusions are clear:
1) Natural immunity to COVID-19 is at least equivalent, if not
superior, to that conferred by vaccines.
2) Administering vaccines to previously infected and recovered
individuals may provide a slim benefit, but one so slim it
would likely be outweighed by the known short-term risks of
vaccination, as well as its unknown potential long-term risks
in young people.
3) The decision to receive added vaccination in the naturally
immune must be left to remain a matter of personal choice
and individual consent, in line with the person’s right to bodily
autonomy, a basic and sacred tenet of medical ethics.
Author Statement
All authors have significantly participated in this work and have
seen and agree with all content of the manuscript
• Ralph Rahme: Conceptualisation, Investigation, Writing-
Original Draft, Writing-Review and Editing
• Aaron Miller: Investigation, Writing-Original Draft
• Sahar Sorek: Investigation, Writing-Original Draft
• Daniel Griepp: Investigation, Writing-Original Draft
• Mahesh Shenai: Investigation, Conceptualisation, Writing-
Review and Editing
• Hooman Noorchashm: Investigation, Conceptualisation,
Writing-Review and Editing
Conflicts of Interest Statement
The authors declare no conflicts of interest.
References
- Bozio CH, Grannis SJ, Naleway AL, Toan C Ong, Kristen A Butterfield, et al. (2021) Laboratory-Confirmed Covid-19 among Adults Hospitalized with Covid-19–like Illness with Infection-Induced or Mrna Vaccine-Induced SARS-COV-2 Immunity — Nine States, January–September 2021. MMWR 70(44): 1539-1544.
- Shenai MB, Rahme R, Noorchashm H, et al. (2021) Equivalency of Protection from Natural Immunity in Covid-19Recovered versus Fully Vaccinated Persons: A Systematic Review and Pooled Analysis. Cureus 13(10): e19102.
- (2021) New CDC Study: Vaccination Offers Higher Protection than Previous COVID-19 Infection. Centers for Disease Control and Prevention 2021.
- Crist C (2021) COVID Vax 5 Times More Protective Than Natural Immunity. WebMD.
- (2021) COVID vaccines five times more effective at preventing COVID-related hospitalization than prior infection alone, study finds. ScienceDaily.
- Slisco A (2021) COVID Vaccines Offer Five Times More Protection Than Immunity From Catching Virus: CDC. Newsweek.
- Green LA (2021) COVID-19: Do vaccines protect better than infection-induced immunity? MedicalNewsToday.
- Mueller B (2021) Vaccination offers more protection against Covid than prior infection, a C.D.C,study suggests. The New York Times.
- Fox M (2021) Vaccination protects against COVID-19 more strongly than previous infection does, CDC study finds. CNN health.
- Dan JM, Mateus J, Kato Y, Kathryn M Hastie, Esther Dawen Yu, et al. (2021) Immunological memory to SARS-CoV-2 assessed for up to 8 months after infection. Science 371(6529): eabf4063 .
- Wajnberg A, Amanat F, Firpo A, Deena R Altman, Mark J Bailey, et al. (2020) Robust neutralizing antibodies to SARS-CoV-2 infection persist for months. Science 370(6521): 1227-1230.
- Kojima N, Klausner JD (2021) Protective immunity after recovery from SARS-CoV-2 infection. Lancet Infect Dis 22(1): 12-14.
- Cohn BA, Cirillo PM, Murphy CC, Krigbaum NY, Wallace AW (2021) SARS-CoV-2 vaccine protection and deaths among US veterans during 2021. Scienc 375(6578): 331-336.
- Levin EG, Lustig Y, Cohen C, Ronen Fluss, Victoria Indenbaum, et al. (2021) Waning Immune Humoral Response to BNT162b2 Covid-19 Vaccine over 6 Months. N Engl J Med 385: e84.
