Abstract

Knowledge of the sensitivities of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody tests beyond 35 days after the clinical onset of COVID-19 is insufficient. We aimed to describe positivity rate of SARS-CoV-2 assays employing three different measurement principles over a prolonged period. Two hundred sixty-eight samples from 180 symptomatic patients with COVID-19 and a reverse transcription polymerase chain reaction (RT-PCR) test followed by serological investigation of SARS-CoV-2 antibodies were included. We conducted three chemiluminescence (including electrochemiluminescence assay (ECLIA)), four enzyme-linked immunosorbent assay (ELISA), and one lateral flow immunoassay (LFIA) test formats. Positivity rates, as well as positive (PPVs) and negative predictive values (NPVs), were calculated for each week after the first clinical presentation for COVID-19. Furthermore, combinations of tests were assessed within an orthogonal testing approach employing two independent assays and predictive values were calculated. Heat maps were constructed to graphically illustrate operational test characteristics. During a follow-up period of more than 9 weeks, chemiluminescence assays and one ELISA IgG test showed stable positivity rates after the third week. With the exception of ECLIA, the PPVs of the other chemiluminescence assays were ≥95% for COVID-19 only after the second week. ELISA and LFIA had somewhat lower PPVs. IgM exhibited insufficient predictive characteristics. An orthogonal testing approach provided for patients with a moderate pretest probability (e.g., symptomatic patients), even for tests with a low single test performance. After the second week, NPVs of all but IgM assays were ≥95% for patients with low to moderate pretest probability. The confirmation of negative results using an orthogonal algorithm with another assay provided lower NPVs than the single assays. When interpreting results from SARS-CoV-2 tests, the pretest probability, time of blood draw, and assay characteristics must be carefully considered. An orthogonal testing approach increases the accuracy of positive, but not negative, predictions.

1. Introduction

Coronavirus Disease 2019 (COVID-19) is a pandemic that has challenged healthcare systems worldwide [1]. The disease is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [2]. Its diagnosis is based on clinical symptoms and signs, radiological imaging, detection of the virus with reverse transcription polymerase chain reaction (RT-PCR) or antigen testing mainly in respiratory specimens, and serological confirmation of SARS-CoV-2-specific antibodies in serum [3, 4]. Although RT-PCR has been reported to possess 100% specificity, it does not provide 100% sensitivity in diagnosing COVID-19 [5], due to issues related to the preanalytical sample quality as well as local and temporal changes in viral shedding [57]. In the clinic, serological testing has increasingly become important to clarify RT-PCR-negative patients with a high clinical suspicion of COVID-19 (“false negatives”) [8]. Furthermore, serological testing has also become important for surveillance and tracing purposes to identify the disease prevalence in a distinct population or identify close contacts of patients with asymptomatic COVID-19 [9].

Recently, a Cochrane review systematically analyzed the sensitivity of different assays in diagnosing COVID-19 and concluded that the time since the onset of symptoms is a critical determinant of test sensitivity [3]. Antibody tests can play a useful role after the first week of symptom onset [3]. However, the authors obtained very little data to describe the sensitivity of the different tests after 35 days. The present study is aimed at describing the positivity rates of different serological tests after a diagnosis of COVID-19 for a prolonged period of up to 9 weeks.

2. Methods

2.1. Study Setting and Study Population

This retrospective study was conducted using anonymized samples. These samples were obtained from patients in Switzerland and the principality of Liechtenstein and sent to a binational group of medical laboratories (labormedizinische zentren Dr. Risch, Switzerland and Liechtenstein). Samples were included in the analysis when patients had a positive RT-PCR for COVID-19 prior to serum sampling. The vast majority of the RT-PCR-positive results was obtained during March and April 2020, and three further RT-PCR-positive results were obtained until June 12, 2020. The serum samples mainly originated from outpatient settings. During March and April, RT-PCR testing in Switzerland and Liechtenstein was offered to individuals presenting with a fever of 38°C and respiratory symptoms, whereas after April 2020, testing was offered to individuals displaying any possible COVID-19 symptom [10]. The study protocol was verified by the cantonal ethics boards of Zurich (BASEC Req-20-00587) and Eastern Switzerland (EKOS; BASEC Nr. Req-2020-00586). Informed consent for performing a laboratory analysis of anonymized samples was waived.

2.2. Laboratory Methods

For each serum sample, the age and sex of the individual, the type of clinical setting, and the number of days from sample collection to RT-PCR were available. Serum employed for testing was either freshly collected or stored for less than 3 months at -25°C. For antibody testing using the chemiluminescence technique, antibodies were tested with three chemiluminescence assays, four enzyme-linked immunosorbent assays (ELISA), and one lateral flow immunoassay, as detailed in Table 1. The manufacturers’ cut-off values as well as the imprecision of the assays are also shown in Table 1: COI (cut-off index) for electrochemiluminescence assay (ECLIA) and S/C (S/C = extinction of the patient sample divided by the extinction of the calibrator) for the other chemiluminescence (i.e., CMIA (chemiluminescent microparticle immunoassay) and LIA (luminescence immunoassay)) and ELISA assays. All tests were performed and analyzed according to the manufacturer’s instructions. All assays were CE marked. Lab technicians reading the lateral flow test format were not aware of the RT-PCR result or the results of other antibody tests when manually reading the lateral flow test cassettes.

2.3. Statistical Methods

Continuous variables are presented as medians and interquartile ranges (IQRs), whereas proportions are presented as percentages and 95% confidence intervals (CIs). The positive rates of the different assays were calculated as fractions of positive results of all investigated samples from patients with positive RT-PCR results (i.e., sensitivity) at different weekly time periods after the first clinical presentation for COVID-19. Statistically significant differences in proportions were tested using the chi-square test. Furthermore, operative test characteristics stratified according to the time point of blood sampling and pretest probabilities were assessed by calculating negative (NPV) and positive (PPV) predictive values [11]. For this analysis, the following specificity values, as evaluated in another cohort investigated by some of the authors of this report in a cohort of 1002 healthcare workers without COVID-19 [12], were employed: 99.5% for the CMIA, 99.7% for the LIA, 99.9% for the ECLIA, 99.2% for the EI IgG ELISA, 99.2% for the EI IgA ELISA, 95.8% for the EDI IgG ELISA, 96.7% for the EDI IgM ELISA, 99.5% for the IgG LFIA, 95.6% for the IgM LFIA, and 95.2% for IgG and/or IgM in the LFIA. For an illustration of predictive values, heat maps were constructed to graphically display predictive values stratified according to the time point of blood sampling and pretest probability. Pretest probabilities were considered very low (1%; e.g., asymptomatic persons in very low prevalence regions), low (<10%; e.g., asymptomatic persons in low prevalence regions), moderate (10-25%; e.g., symptomatic persons with a suspicion of COVID-19), and high (>25%; e.g., symptomatic household contacts of patients with COVID-19) [10]. In addition to the predictive values for single tests, we also evaluated orthogonal testing algorithms that sequentially combined the test characteristics of two tests according to the method proposed by the Food and Drug Administration (FDA) and the Centers of Disease Control (CDC) in the United States of America [13, 14]. Predictive values were rounded to integers. values < 0.05 were considered statistically significant. Statistical computations were performed with MedCalc version 18.11.3 (Mariakerke, Belgium). Graphs were drawn with Microsoft Excel 2016 MSO (16.0.8431.2046) (Microsoft Inc., Seattle, USA) using the linear interpolation function.

3. Results

3.1. Sensitivity

Two hundred sixty-eight samples from 180 patients were available. The median age at diagnosis was 56 years IQR [38, 72]. Not all samples were tested with all tests. The number of tested samples and the number of positive samples are presented in Table 2. The rates of positive samples over the different weeks are illustrated in Figure 1. Figure 1(a) shows the positivity rates of the chemiluminescence assays, and all three investigated formats had the highest positivity rate during the 3rd week after the clinical presentation. After this period, the positivity rates for all three investigated assays decreased to approximately 92% and remained stable. For the ELISAs, a peak was also observed at week 3 for IgG and IgM, whereas the peak for IgA occurred at week 5. IgG exhibited a different behavior, according to the test manufacturer. Although the EI positivity rate remained constant after 9 weeks, the positivity rate of EDI decreased by approximately 40%. The IgM positivity rate decreased even further to approximately 15%. IgA also showed a decrease in the positivity rate to approximately 80%. In the LFIA, IgM showed an earlier increase in positivity rates than IgG, with IgG and IgM positivity rates peaking at a value of approximately 80% at week 3. Notably, the LFIA displayed a significantly higher sensitivity than the LIA, the EI IgG ELISA, and the EDI IgM ELISA (78 vs. 50%, ; vs. 36%, ; vs. 53%; ) at week 2. The positivity rate of IgM decreased to 30% after 9 weeks, whereas the IgG positivity rate decreased to 70%. The LFIA test format reporting combined reaction of both IgG and IgM showed a positivity rate of approximately 90% at week 3, which decreased to somewhat lower values at week 9.

3.2. Predictive Values of Single Tests

Predictive values were calculated as a function of pretest probabilities encountered in the clinical routine (i.e., 1% to 40%) to determine the effect of the different diagnostic characteristics at different time points of blood sample collection on the operational test characteristics [10]. Figure 2 illustrates the positive predictive values (PPVs) at different time points, whereas negative predictive values (NPVs) are presented in Figure 3.

The chemiluminescence assays displayed good predictive characteristics in excluding COVID-19 with a negative test result if the sample was collected at week 3 or thereafter. In situations with a moderate to high pretest probability, none of the chemiluminescence assays would be capable of excluding a COVID-19 infection during the first two weeks of a suspected infection. A positive test result obtained using CMIA and LIA for asymptomatic patients with a very low and low pretest probability had a PPV of less than 95%, regardless of the time point at which the blood sample was collected. Positive results obtained using the ECLIA at various sampling time points from patients with a low pretest probability of at least 5% had a for the prediction of COVID-19.

Regarding the ELISAs, the operational test characteristics of the assay from one manufacturer are better than the assay from another manufacturer. The EDI IgG and IgM ELISAs, however, generally showed low PPVs during the first two weeks and thereafter in low pretest probability settings, independent of the blood sampling time point. The ability of the EI ELISAs to exclude COVID-19 during the first two weeks with an NPV of 95% or greater is possible only in patients with very low and low pretest probabilities. Thereafter, the exclusion of COVID-19 with a negative test result was able to be relatively safely conducted in samples from patients with higher pretest probabilities. The EDI IgG ELISA excluded COVID-19 with for patients with very low, low, and moderate pretest probabilities, whereas the IgM assay cannot be reliably used in this context. Regarding a positive diagnosis of COVID-19, neither EDI ELISAs appeared to add valuable information in any situation when used as single tests.

The LFIA IgG measurement appeared to possess the potential for diagnosing COVID-19 in patients with moderate and higher pretest probabilities, but not in patients with very low and low pretest probabilities, when blood samples were collected at least 3 weeks after symptom onset. Positive LFIA IgM results did not display a useful PPV in predicting COVID-19. The combined judgment of both IgG and IgM LFIA tests was not useful in reliably predicting COVID-19. The combined LFIA test possessed an NPV of ≥95% in excluding COVID-19 in patients with very low and low pretest probabilities (i.e., symptomatic and asymptomatic patients), regardless of the time of sampling. At week 3 or thereafter, the NPV also reached values of ≥95% for patients with moderate pretest probabilities. Together, the LFIA appeared to exclude COVID-19 in patients with low pretest probabilities throughout the disease course with negative combined IgG/IgM results. Furthermore, LFIA IgG tests also excluded COVID-19 in patients with a moderate pretest probability at week 3 after the first clinical presentation and thereafter.

3.3. Orthogonal Testing Algorithms

In addition to the IgM ELISA, all investigated tests provided relatively high NPVs in excluding COVID-19 in patients with negative test results and low to moderate pretest probabilities over the whole 9 weeks. With the exception of the ECLIA, the PPVs were less than 95% for individuals with low pretest probabilities over the whole 9 weeks. Chemiluminescence tests for patients with a moderate pretest probability provided , whereas the PPVs for ELISAs and LFIA were lower. We therefore determined the PPVs using an orthogonal testing approach, where positive results were confirmed with another independent test. The combination of two of the lowest performing tests for positive results in an orthogonal testing algorithm, i.e., the LFIA IgG/IgM test followed by the EDI IgG ELISA test, for patients with a moderate pretest probability consistently provided PPVs of ≥95%, as shown in Figure 4. A combination of two chemiluminescence tests (i.e., CMIA followed by LIA) provided PPVs of 100% for patients with all pretest probabilities (Figure 4). Other test combinations (i.e., ECLIA/EDI IgG, ECLIA/LIA, ECLIA/CMIA, ECLIA/EI IgG, CMIA/EDI IgG, CMIA/EI IgG, LIA/EDI IgG, and LIA/EI IgG) provided comparable PPVs (≥99%) to the CMIA/LIA combination throughout the whole 9-week period for patients with all pretest probabilities (data not shown). NPVs of a CMIA/LIA combination were lower than those of the CMIA or the LIA alone, as shown in Figure 5. Other combinations of negative test results exhibited similar performance and are not shown.

4. Discussion

A recent Cochrane review stated that data available for diagnostic characteristics of SARS-CoV-2 antibody assays for more than 35 days after symptom onset are insufficient [3]. The present study provides data for a variety of tests beyond 9 weeks (63 days). We confirm that the investigated tests have insufficient test characteristics, particularly for positive results, two weeks after the first clinical presentation, with one exception. The combination of tests in an orthogonal testing algorithm, even tests with suboptimal test characteristics, for positive results provided for individuals with moderate and high pretest probabilities early in the course of the infection. Furthermore, positivity rates appeared to be stable for chemiluminescence assays testing for total Ig and IgG, whereas the positivity rates of IgM tests decreased over time.

Fenwick and colleagues and others observed sensitivities than the present study in the ECLIA (95.6%), LIA (88.9%), EI IgG (88.9%), and EDI IgG (76.7%), which corroborates our findings derived in smaller cohorts stratified according to time after the first clinical presentation of COVID-19 [1518]. The sensitivity of the CMIA described by Tang et al. (93.8%) is comparable to the sensitivity values observed in our study [18]. The sensitivity of the LFIA was also comparable to the values reported for other LFIA test formats (75% in the second week and 90% thereafter) [19]. Some test formats analyzed in our study had stable sensitivities over a duration of 9 weeks after the first clinical presentation. Other test formats, particularly tests measuring IgM and IgA, as well as an ELISA measuring IgG directed against the nucleocapsid antigen, showed decreasing positivity rates over time. We consider the use of test formats with decreasing sensitivities over time problematic, as long as they do not employ modified cut-off values, which should be further investigated in larger cohorts.

In clinical diagnostics, the operational characteristics rather than the diagnostic characteristics are of interest. In our opinion, the use of heat maps to illustrate diagnostic strengths and weaknesses in certain situations according to the time since the first clinical presentation and pretest probability is useful, particularly when only one test is employed. To the best of our knowledge, heat maps have not yet been introduced as a tool to interpret serological assays of infectious diseases [4]. Unfortunately, at the moment, there is no clinical score currently available to assess pretest probability, which relies on a history of clinical symptoms alone and could also be utilized retrospectively (e.g., because no laboratory results are available at the time when a patient was symptomatic) [20]. However, other rough estimates also allow for assessment of pretest probability and clinical use of the heat maps according to the individual situation of a patient: symptomatic patients have a pretest probability of 10% and higher, asymptomatic patients usually have a pretest probability of less than 10%, and close contacts of patients with confirmed COVID-19 cases have a pretest probability of 15-30% [10]. Seroprevalence data of a region can also help to assess the pretest probability even in the absence of clinical symptoms.

Positive predictive values differed among the different test formats. Based on the results of our study, chemiluminescence formats have somewhat better operative characteristics than at least some of the investigated ELISA and LFIA test formats. Furthermore, the SARS-CoV-2 IgM test should be used with caution to assess an immune response to COVID-19 that has already been apparent for longer than 21 days. This result is consistent with the findings of Coste et al., who, because of the simultaneous occurrence with IgG, judged IgM to be of no value in diagnosing acute or subacute SARS-CoV-2 infections [21]. Our study clearly illustrates that an orthogonal testing approach adds valuable diagnostic information. The combination of two of the lowest performing assays according to the approach recommended by FDA and CDC for moderate pretest probabilities provided throughout the 9 weeks after the first clinical presentation. These characteristics further improved and were extended to low prevalence settings when the chemiluminescence and EI IgG assays were combined. Our study confirms that an orthogonal testing approach for initially positive samples adds diagnostic value.

After two weeks, negative predictive values of tests investigating IgG or total Ig were ≥95% in patients with very low, low, and moderate probabilities. Chemiluminescence assays displayed a somewhat higher predictivity than ELISAs and LFIA. Importantly, the LFIA reading for both IgM and IgG bands showed greater predictive value than reading either of the two bands alone. However, in contrast to the practice of confirming positive test results with an alternative test, the application of an orthogonal testing approach is not useful in a sample with an initially negative test result. NPVs were worse than the initial index test.

Our study has several strengths and limitations. One strength is that we were able to study positivity rates over 9 weeks. Another strength is that we studied 8 different assays employing the commonly used measurement principles, i.e., chemiluminescence, ELISA, and lateral flow immunoassay. A limitation is that the day of onset of clinical symptoms was not available. Instead, we recorded the first date of clinical presentation for COVID-19. In Liechtenstein and Switzerland, all symptomatic patients are tested for COVID-19 using PCR. Due to the high accessibility of testing in COVID-19 testing centers, patients presented to these centers very rapidly after the onset of symptoms. Additionally, our data, particularly the results obtained within the first 35 days, are similar with the findings described elsewhere using the onset of symptoms as the reference time point[3, 18, 22]. Some studies have also presented data using PCR testing as a reference time point [18, 22]. We therefore propose that our approach of using the first clinical presentation for COVID-19 testing in the investigated setting is a good proxy for the onset of symptoms. When analyzing the different time strata, the sample size of the included samples was not large, and not all sera were tested with all tests, which might result in relatively imprecise point estimates of sensitivity. The sample sizes are comparable to the values reported in other studies, and the sensitivities identified in our setting are comparable to the values reported in larger other studies, suggesting that relatively small sample sizes do not invalidate our findings [3, 15, 17, 18, 21, 22]. Finally, even if we did not have a detailed clinical description of the patients, we know that the samples of our patients originated from outpatient settings. Thus, it can be assumed that the included confirmed symptomatic patients were with mild to moderate COVID-19 severity. Our findings might not apply to individuals with severe cases and an asymptomatic disease course [15, 23], which requires confirmation in further investigations. Our findings, however, provide valuable insights into interpreting past symptoms in patients who were not assessed using RT-PCR testing or who had a negative RT-PCR test, despite the presence of suggestive symptoms, at least within two months of clinical presentation.

In conclusion, we identified sensitivities for 8 different SARS-CoV-2 antibody tests in symptomatic patients over a period of 63 days. The investigated chemiluminescence assays did not exhibit a decrease in positivity rates over the investigated period. This finding was not observed for other test formats, e.g., IgM tested using ELISA or LFIA. Not all commercially available and CE-marked assays exhibit satisfactory predictive values for diagnosing or excluding COVID-19. In particular, predictive values were lower during the first two weeks after the first clinical presentation. An orthogonal test algorithm confirming positive, but not negative, results from a single test with an independent assay provided satisfactory positive predictive values, even for tests with less accurate performance. Orthogonal testing approaches for positive results should therefore be reinforced.

Data Availability

The data used to support the findings in this study will be available from the corresponding author upon request.

Disclosure

The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Conflicts of Interest

The authors declare no conflicts of interest.

Authors’ Contributions

Martin Risch was responsible for conceptualization, methodology, formal analysis, and writing of the original draft; Myriam Weber was responsible for conceptualization, methodology, and formal analysis; Sarah Thiel was responsible for data curation, validation, and resources; Kirsten Grossmann was responsible for resources; Kirsten Nadja Wohlwend was responsible for investigation, validation, and resources; Thomas Lung was responsible for validation and resources; Dorothea Hillmann was responsible for investigation, validation, project administration, and resources; Michael Ritzler was responsible for resources; Francesca Ferrara was responsible for resources; Susanna Bigler was responsible for validation and resources; Konrad Egli was responsible for validation and resources; Thomas Bodmer was responsible for validation and resources; Mauro Imperiali was responsible for validation and resources; Yacir Salimi was responsible for validation and resources; Felix Fleisch was responsible for resources; Alexia Cusini was responsible for resources; Harald Renz was responsible for conceptualization; Philipe Kohler was responsible for validation and writing of the original draft; Pietro Vernazza was responsible for conceptualization and writing of the original draft; Christian Kahlert was responsible for conceptualization, methodology supervision, resources, and writing of the original draft; Matthias Paprotny was responsible for supervision and resources; Lorenz Risch was responsible for conceptualization, methodology, funding acquisition, supervision, resources, and writing of the original draft. All authors were involved in writing, reviewing, and editing.

Acknowledgments

The assistance of Toni Schönenberger and Walter Frehner in identifying samples is acknowledged. The research project was funded by a grant from the government of the Principality of Liechtenstein.