Clinical assessment of SNIBE Maglumi SARS-CoV-2 antigen fully-automated chemiluminescent immunoassay

Introduction

The ongoing coronavirus disease 2019 (COVID-19) pandemic represents a paramount challenge for many political, social, clinical and even diagnostic reasons. As concerns the last issue, a scarcely prepared and frequently inefficient laboratory response to COVID-19 has been widely evidenced over the course of the pandemic [1], wherein the organization of many clinical laboratories around the world was disrupted by the immense number of tests needed for diagnosing severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection [2]. Although molecular testing, specifically real-time reverse transcription polymerase chain reaction (RT-PCR), remains the gold standard technique for detecting and even quantifying SARS-CoV-2 RNA, this method has some important limitations, including limited availability especially in some worldwide regions (i.e., mostly in medium-to-low income countries), shortage of reagents, long turnaround time (a single RT-PCR test can take several hours to generate results), need for trained staff and specialized laboratory instrumentation [3]. To overcome some of these technical shortcomings, the World Health Organization (WHO) [4] and the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) [5] have both recently endorsed the usage of SARS-CoV-2 antigen testing in certain clinical settings, namely for large (mass) screening and for the identification of highly infective individuals, who are responsible for the largest number of contagions [6]. Evidence has also been provided that the integration of SARS-CoV-2 molecular and antigen testing, especially when the latter encompasses laboratory immunoassays, may be even more cost-effective than RT-PCR alone [7]. Thus, rapid, accurate and relatively inexpensive SARS-CoV-2 antigen tests are now seen as a pillar for surrogating molecular testing in the next (predictably long) endemic period of COVID-19 [8, 9]. Nevertheless, one key aspect that shall drive the introduction of SARS-CoV-2 antigen immunoassays, either rapid (i.e., RDT-Ag) or laboratory based, within the routine clinical practice, is that their analytical performance and diagnostic accuracy must be independently verified and validated in clinical laboratories [4, 5]. We have hence assayed here the clinical performance of the new SNIBE Maglumi SARS-CoV-2 antigen fully-automated chemiluminescent immunoassay (MAG-CLIA) for diagnosing acute SARS-CoV-2 infection.

Materials and methods

Results

The final study population consisted of 181 consecutive unselected individuals (mean age 61 ± 21 years; 92 females). A total number of patients 118 (65%) tested positive for SARS-CoV-2 RNA with molecular testing, 84 (46%) displaying a mean Ct value <29.5. The median Ct value in positive samples was 25.2 (IQR, 21.2–31.5) for SARS-CoV-2 E gene, 24.6 (IQR, 21.0–30.8) for SARS-CoV-2 S gene, and 24.9 (IQR, 21.1–31.2) for the mean of the two SARS-CoV-2 genes. A total number of 77 patients (43%) “were reactive” (i.e., >25 ng/L) with SARS-CoV-2 MAG-CLIA, whilst 84 (46%) had quantifiable (i.e., 4 ng/L) test results with SARS-CoV-2 MAG-CLIA.

A significant correlation was found between Ct values and N protein concentration measured with MAG-CLIA SARS-CoV-2 Ag in the RT-PCR positive samples (r=−0.95; 95% CI, −0.97 to −0.93; p<0.001). In ROC curve analysis, the AUC of MAG-CLIA SARS-CoV-2 Ag was 0.86 (95% CI, 0.81–0.90) for identifying positive samples and 0.98 (95% CI, 0.96–1.00) for detecting those with high viral load (i.e., mean Ct value <29.5). At the 25.0 ng/L cutoff recommended by the manufacturer, the accuracy was 77.4% (95% CI, 70.1–83.2%), the sensitivity was 0.65 (95% CI, 0.56 to 0.74), the specificity was 1.00 (95% CI, 0.94–1.00), the negative predictive value (NPV) was 0.61 (95% CI, 0.54–0.63) and the positive predictive value (PPV) was 1.00 (95% CI, 1.00–1.00). The best cutoff calculated from the ROC curve analysis was 7.0 ng/L, associated with 81.2% (95% CI, 74.8–86.7%) accuracy, 0.71 (95% CI, 0.62–0.79) sensitivity, 1.00 (95% CI, 0.91–1.00) specificity, 0.65 (95% CI, 0.58–0.71) NPV and 1.00 (95% CI, 1.00–1.00) PPV for identifying SARS-CoV-2 positive samples (Figure 1 and Table 1). Samples with high viral load were also best detected using the 7.0 ng/L cutoff, which was associated with 96.7% (95% CI, 92.9–98.8%) accuracy, 0.96 (95% CI, 0.90–0.99) sensitivity, 0.97 (95% CI, 0.91–0.99) specificity, 0.97 (95% CI, 0.91–0.99) NPV and 0.96 (95% CI, 0.90–0.99) PPV, respectively (Figure 1 and Table 1).

Figure 1:

Receiving Operating Characteristic (ROC) curve analyses of SNIBE Maglumi SARS-CoV-2 antigen fully-automated chemiluminescent immunoassay for diagnosing acute SARS-CoV-2 infection. 95% CI, 95% confidence interval; AUC, area under the curve.

Importantly, when SARS-CoV-2 N protein concentration was replaced with raw instrumental readings (i.e., RLU), the AUC in all samples increased to 0.94 (95% CI, 0.90–0.97; p<0.001), enhanced further to 0.99 (95% CI, 0.98–1.00) in those with high viral load (Figure 1). A RLU value of 945 was associated with 88.4% (95% CI; 82.8–92.3%) accuracy, 0.85 (95% CI, 0.77–0.91) sensitivity, 0.95 (95% CI, 0.87–0.99) specificity, 0.77 (95% CI; 0.68–0.84) NPV and 0.97 (95% CI; 0.92–0.99) PPV for diagnosing acute SARS-CoV-2 infections. A 945 RLU cut-off enabled to identify samples with high viral load with 1.00 (95% CI, 0.96–1.00) sensitivity and 0.80 (95% CI, 0.71–0.88) specificity.

Discussion

It is now undeniable that the COVID-19 global pandemic has generated a tremendous impact on many human domains, principally on health, society and economy. As the virus continues its worldwide surge, now characterized by an attenuated pathogenicity due to accumulation of many mutations in its genome combined with growing immunity (both natural and vaccine-elicited) [14], the rapid and accurate identification of people infected by SARS-CoV-2 remains vital, especially of those bearing high viral load in the upper respiratory tract, who may further contribute to spread the virus and thus infect the more vulnerable part of the population, enhancing the risk of a recrudescence of severe/critical forms of COVID-19 illness as in the earlier periods of this pandemic.

Antigen immunoassays are a type of COVID-19 test encompassing the detection of SARS-CoV-2 viral antigens in respiratory samples, especially nasal, nasopharyngeal and/or oropharyngeal swabs. Laboratory based SARS-CoV-2 antigen immunoassays are conceptually similar to RDTs, though the former techniques display higher analytical and diagnostic sensitivity, which is globally reflected by a better diagnostic accuracy, as shown by the meta-analysis recently published by Tapari et al. [15], who evidenced that the diagnostic sensitivity of RDT-Ag is nearly 10% lower compared to that of laboratory-based antigen immunoassays. In a fast-moving environment, characterized by unremittent development and commercialization of new SARS-CoV-2 diagnostic tests, one mainstay before using such methods in clinical practice is the need to conduct a preliminary verification and validation of both analytical and clinical performance, with the aim to verify that manufacturer’s claims and minimum performance criteria can be met [16]. To this end, we planned this study for assessing the clinical performance of the new quantitative and high-throughput MAG-CLIA SARS-CoV-2 Ag.

Although we certainly failed to reproduce the otherwise very optimistic diagnostic performance observed by Wang et al. [10], who reported sensitivity and specificity as high as 0.96 and 0.99 in their study using a 0.64 ng/L cutoff, the global accuracy of the MAG-CLIA SARS-CoV-2 Ag was actually found to be comparable to that reported for other similar immunoassays [17]. In particular, the sensitivity at the locally calculated cutoff (i.e., 7.0 ng/L) was very similar to the pooled sensitivity of other commercial SARS-CoV-2 Ag immunoassays (i.e., 0.71 vs. 0.73), whilst the specificity was even better (i.e., 1.00 vs. 0.98). In samples with high viral load, the sensitivity was found to be instead much better than that observed with other similar SARS-CoV-2 Ag immunoassays (i.e., 0.96 vs. 0.87), thus making MAG-CLIA SARS-CoV-2 Ag a very efficient method for identifying subjects with high viral load, with accuracy comparable to that of other “high-sensitive” SARS-CoV-2 Ag immunoassays [18, 19]. Nonetheless, one important aspect that emerged from this study, is that when we replaced SARS-CoV-2 N protein concentration with raw instrumental readings (i.e., by using instrumental RLU), the diagnostic performance of MAG-CLIA SARS-CoV-2 Ag displayed a considerable improvement, with the AUC increasing to 0.94, the accuracy to 88.4%, the sensitivity to 0.85, the specificity to 0.95, the NPV to 0.77 and the PPV to 0.97, respectively. This is not really surprising, as Wang et al. found that the LoQ of this immunoassay was 0.4 ng/L, thus much lower than that reported by the manufacturer in the Instructions for Use (IFU; i.e., 12 ng/L). Although the best RLU cutoff that we calculated (i.e., 945) corresponded to an antigen concentration lying below the reportable range of the assay (i.e., <4 ng/L), we extrapolated from the calibration curve that the corresponding SARS-CoV-2 N protein value could have been around 1.9 ng/L, i.e., twice as low as the LoB reported by the manufacturer, and nearly 3-fold higher than the best diagnostic threshold identified by Wang et al. (i.e., 0.64 ng/L) [10]. To this end, we proffer that the manufacturer should consider revising its current IFU, broadening the range of reportable values to embrace lower SARS-CoV-2 N protein concentrations. In keeping with the findings of Wang et al. [10], the use of a lower cutoff is in fact likely to yield improved diagnostic accuracy.