Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused over 108 million cases of coronavirus disease 2019 (COVID-19) and over 2.5 million deaths globally [1]. Many COVID-19 diagnostic tests are now available. These include molecular assays such as real-time reverse transcription polymerase chain reaction (rRT-PCR) and rapid antigen tests which are useful for identifying acute SARS-CoV-2 infections [2]. Serologic assays can detect antibodies that can be produced after infection or vaccination, as well as used to understand the presence of antibodies across a population [3].

Specific serologic assays include enzyme-linked immunosorbent assays (ELISA), chemiluminescent immunoassays, neutralization assays, and lateral flow immunoassays [4]. Currently, the SARS-CoV-2 spike (S) and nucleocapsid (N) proteins are the most commonly used targets in serological assays [5]. Natural infections in adults with SARS-CoV-2 should induce antibodies both to the S and N proteins [6]. However, antibody responses to vaccination depend on the antigen proteins within (or induced by) a vaccine formulation. Different assays may also have variable limits of detection and may detect different antibody isotypes (e.g., Immunoglobulin (Ig) A or IgA, IgM, IgG) which can rise at different rates following infection. However, knowledge on SARS-CoV-2 antibody kinetics is limited, which has challenged optimization of test usage and interpretation [7].

To understand how the performance of antibody serologic assays may differ, we evaluated antibody responses in SARS-CoV-2 cases and their household contacts using two assays: the SARS-CoV-2 S-Protein IgG ELISA (spike ELISA) [8] and the Abbott Architect^TM SARS-CoV-2 IgG assay (Architect) [9]. Additionally, we investigated whether patient characteristics were associated with differences in assay results.

Materials and methods

Results

Ninety participants including 26 index cases and 64 contacts from 26 households were enrolled. One hundred five serum samples were collected from 73 unique individuals across 23 households, including 23 index cases and 50 contacts. Fifty-six of 105 serum samples were collected at visit 1, while 49 were collected at visit 2 (i.e., 14 days later). Of all serum samples, 40 (38.1%) were concordant positive, 58 (55.2%) were concordant negative, and seven (6.7%) were discordant. The seven discordant serum samples were from unique individuals and all seven tested positive by spike ELISA and negative by Architect. The overall kappa coefficient value for all serum samples was 0.86, suggesting strong agreement between the two assays.

Table 1 presents demographic, clinical, and laboratory characteristics of the 47 individuals who had positive serology results for one or both assays. Among the 47 respiratory samples collected from these individuals before or during the study period, 43/47 (91.4%) were from individuals who tested positive by rRT-PCR at or before the time of serum collection. Of these 37 (86%) had a positive rRT-PCR result more than 7 days prior to serum collection, while six (14%) were rRT-PCR-positive within 7 days of serum collection. Fisher’s exact test showed that none of the clinical variables, including time from symptom onset, were statistically significantly different between concordant positive and discordant samples.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Demographic, clinical, and laboratory characteristics of the 47 serum samples tested by both SARS-CoV-2 spike protein ELISA (spike ELISA) and Abbott Architect^TM IgG assay (Architect) with at least one positive serology assay result.

https://doi.org/10.1371/journal.pone.0255208.t001

A detailed description of the seven individuals with discordant results is presented in Table 2. The age of the individuals with discordant results ranged from 15–54 years and included both male (n = 5) and female (n = 2) persons. The days from symptom onset to time of sample collection for serologic testing ranged from 0–29 days, and one person was asymptomatic. All the seven discordant samples were positive by spike ELISA and negative by Architect. Six of the seven individuals had a positive rRT-PCR test during the investigation and the time between first rRT-PCR positive test and serum collection ranged from 0 to 29 days.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Detailed description of individuals with discordant serology assay test results (n = 7).

https://doi.org/10.1371/journal.pone.0255208.t002

Signals for both the assays were plotted for all 105 serum samples (Fig 1). The IgG S/T value of the CDC ELISA for the seven discordant samples ranged from 1.5 (weak positive) to 6.5 (strong positive), with a mean of 3.4 and 0.45 coefficient of variance. Among the serum samples with concordant positive results the mean IgG S/T value of the CDC ELISA was 5.7 and the mean S/C value of the Architect assay was 6.1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Comparison of the IgG signal detected by the SARS-CoV-2 spike protein ELISA (spike ELISA) and the Abbott Architect^TM SARS-CoV-2 IgG assay (Architect) (n = 105 serum samples).

IgG signal to threshold (S:T) value only for Spike ELISA and the Abbott Architect^TM SARS-CoV-2 IgG (Architect) assay using 105 serum samples. For the Spike ELISA, S/T value was calculated by dividing the OD for 1:100 dilution by 0.4; and, for Architect the system calculates a calibrator mean chemiluminescent signal and the default result unit is index (S/C). Cut-off value was 1 and 1.4 for Spike ELISA and the Architect, respectively. The 105 samples were differentiated into: concordant positive (n = 40), concordant negative (n = 58) and discordant (n = 7).

https://doi.org/10.1371/journal.pone.0255208.g001

Among serum samples from individuals with positive rRT-PCR results, IgG signal increased with days since symptom onset for both the anti-nucleocapsid protein and anti-s protein-based assays (Fig 2). Sensitivity of the two assays is influenced by time since symptom onset and time since exposure (data not shown).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Comparison of SARS-CoV-2 spike protein ELISA (spike ELISA) and Abbott Architect^TM SARS-CoV-2 IgG assay using serum samples from individuals with a positive rRT-PCR test result (n = 52) by days since symptom onset.

*Each blue circle (O) represents a serum sample tested by the Architect assay; each red cross (X) represents a serum sample tested by the Spike ELISA. Samples collected from asymptomatic individuals (n = 4) were excluded from the analysis.

https://doi.org/10.1371/journal.pone.0255208.g002

Discussion

We compared side-by-side results from two serologic assays, one that detects antibodies to the S- protein and one that detects antibodies to the N-protein of SARS-CoV-2 virus. Our results showed a concurrence of 93.3%, suggesting strong agreement between the two assays when used in patients to detect a history of infection with SARS-CoV-2.

In this study, all discordant samples (n = 7) were positive by the spike-protein-based assay (Spike ELISA) and were negative by the nucleocapsid-based immunoassay (Architect). Our findings are similar to a recent study conducted by Liu et al., where an spike-protein-based ELISA was found to be more sensitive than an nucleocapsid-based assay [14]. The predetermined quantitative threshold of a qualitative positive test as identified by the manufacturer might also vary from one assay to another. The discordance could therefore be attributed to inherent variations in assay design, target antigens, or expected differences in individuals’ polyclonal antibody responses in the timing of specific antibodies generated from infection [15].

Studies have shown that seroconversion for IgG occurs earlier for SARS-CoV-2 anti-N antibodies than for anti-S antibodies and IgG reactivity against SARS-CoV-2 N antigen was detectable by 2 weeks after symptoms onset [16–18]. This is similar to data from SARS-CoV-1 and other human coronavirus infections, in which antibodies against the N-protein were detected significantly earlier than antibodies to the S-protein [19]. Nucleocapsid (N) is produced in abundance during the early phases of viral replication [20]. If viral replication is not controlled well by the innate or early T cell immune responses, N could be abundant and therefore induce a robust antibody response [21]. However, if replication is well controlled and the immune system primarily sees intact virions, the antibody response might be spike dominant. These findings demonstrate that serologic assays face limitations inherent to the human immunologic response, and may therefore provide some discrepant results even when testing the same samples across different assays [22].

Further studies are needed to understand how patient and illness characteristics may result in variability among immunoassay performance, including shortly after infection (<14 days after symptom onset). Given the overall sample size and small number of discordant results, this analysis lacked statistical power to evaluate these characteristics by assay performance. It has been proposed that serologic testing could be reliably used after 14 days post symptom onset [23], since sensitivity within the first 14 days of symptom onset is highly variable for most SARS-CoV2 antibody assays but improves after 14 days [9, 24]. Analyses on serologic assay performance using sera collected <14 days of symptom onset provide a clearer picture of assay performance during the early phase of disease.

Currently, in the United States, the three authorized COVID-19 vaccines are based on the SARS-CoV-2 spike protein [25]. As vaccines become universally available, differentiating SARS-CoV-2 antibodies induced by vaccination versus natural infection may provide valuable epidemiologic information. Multiplex serology testing developed to measure antibody response to more than one antigen might be useful to monitor asymptomatic infection rates and assess for reinfection, vaccine breakthrough, and population-level immunity achieved by either natural infection or vaccination. Because current US-authorized vaccines utilize S protein antigens only, serology tests that assess spike protein antibodies can be used to evaluate immune response to vaccination, whereas those targeting nucleocapsid protein antibodies could serve as markers for natural infection.

In conclusion, our data indicates that both the spike ELISA and Architect assay had high agreement, but rarely paired tests results can vary when using two different serologic assays. Choosing a serologic assay requires consideration of current and past incidence of COVID-19 in a geographic area, characteristics of the targeted patient population to be tested, vaccination history and required performance characteristics of the test.

Supporting information

Acknowledgments

We would like to acknowledge the contributions of Zorangel Amezquita-Montes, Noah Leigh, Joshua Weiner, and Trivikram Dasu from the City of Milwaukee Department of Health Laboratory; and Elizabeth Rabold, Rebecca Laws, Radhika Gharpure, Erin Conners, Patrick Dawson, and Scott Nabity from the CDC’s COVID-19 Response Epidemiology and Surveillance Task Force, for their meaningful contributions in support of the investigation.

Disclaimer: The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the Centers for Disease Control and Prevention or the institutions with which the authors are affiliated.