SARS-CoV-2 serologic tests: do not forget the good laboratory practice

To the Editor,

Although the detection of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) RNA in respiratory specimens remains the diagnostic standard for COVID-19, SARS-CoV-2 antibody testing has been advocated for identifying individuals who suffered infection. From April 2020, when the first assay for SARS-CoV-2 antibodies came to the limelight, several immunoassays have been marketed. Simultaneously, plenty of papers were published focusing on the clinical validation of these tests and there is still debate regarding which immunoassay type should be used, which antibody types should be measured, and which viral protein domain is more informative [1], [2]. It is however embarrassing to note the extremely low number of papers that at the same time dealt with the real-life analytical performance of SARS-CoV-2 serologic assays. This journal has attempted to promote the subject, but with scarce and preliminary contributes [3], [4], [5]. So that, as laboratory professionals and, more in general, as healthcare providers we are still unable to answer the significant question of whether the analytical quality of results provided by serologic assays is fit-for-purpose. When not properly evaluated and monitored they have indeed the potential to misdiagnose and misinform [6].

Measurement uncertainty (MU) represents a key quality indicator of both the performance of an IVD measuring system (MS) and the laboratory itself [7]. Medical laboratories should estimate and validate the MU of each test to improve the quality of provided results and, ultimately, patient safety. Here, we estimated MU of a SARS-CoV-2 antibody assay due to random error (u_Rw), which is often the largest contributor to the MU of measured quantity values on clinical samples.

The Liaison SARS-CoV-2 S1/S2 IgG test (DiaSorin) is a two-step chemiluminescence immunoassay, performed on the Liaison analyzers. Results are expressed as arbitrary units (AU)/mL and, for the CE-marked assay, are negative if <12 AU/mL, uncertain if between 12 and <15 AU/mL, and positive if ≥15 AU/mL. Interestingly, the result interpretation, as recommended by the manufacturer in Europe, differs from that included in the U.S. Food and Drug Administration approved version, for which results are negative if <15 AU/mL and positive if ≥15 AU/mL, and no “grey-zone” is expected. ISO 20914:2019 indicates how to obtain u_Rw, which represents the uncertainty component under within-laboratory intermediate reproducibility conditions (i.e. the uncertainty for a given MS that includes routine changes to measuring conditions, e.g., reagent lot changes, calibrations, instrument maintenance, etc.) [8]. Internal quality control (IQC) measurements to evaluate u_Rw should therefore be performed over a sufficiently extended period, able to cover all above-reported changes [9]. IQC materials used to estimate u_Rw should be from a third-party source, independent from the MS manufacturer, be commutable and have an analyte concentration relevant to the clinical application of the measurement [7], [9]. Since commercial control materials for anti-SARS-CoV-2 IgG determination with these characteristics are not available, we prepared a serum pool with an anti-SARS-CoV-2 IgG concentration around the target value of 15 AU/mL, manufactured with anonymized leftover samples and stored aliquoted at −20 °C. From May to September 2020, we measured anti-SARS-CoV-2 IgG on a Liaison XL platform using a freshly thawed pool aliquot every weekday. The MS alignment to the manufacturer’s specifications was checked by measuring the two-level control material offered by DiaSorin as part of their CE-marked MS (ref. 311461). After a preliminary experience by using the validation range recommended by the manufacturer (target ± 50%), we restricted the range of positive control to ±33% to better highlight shifts due to poor calibration. u_Rw was estimated as the mean of monthly CVs during the entire study period.

Monthly CVs ranged from 8.0 to 13.6%, with an average CV of 11.6% (Table 1). By applying a coverage factor of 2 (95% level of confidence), the relative expanded MU of SARS-CoV-2 S1/S2 IgG measurement on the Liaison was 23.2% at cut-off threshold. Tré-Hardy et al. [4] previously validated the Liaison test. They reported an assay reproducibility from 4.7 to 5.1%, in line with DiaSorin data, but markedly lower than that found in our study. Both manufacturer’s and authors’ precision protocols were however performed in a very short period of time (days or weeks) and in strictly controlled conditions, which portray an optimistic estimate of imprecision and fail to reproduce real-life measuring conditions in medical laboratories, therefore underestimating random sources of MU. Moreover, to correctly estimate u_Rw a commutable material from a third-party source must be used [9]. For testing imprecision, Tré-Hardy et al. used however the same control material provided by the manufacturer for checking the MS alignment, not validated for commutability. Characteristics of IQC material for estimating u_Rw have been defined and should be carefully considered, i.e. the material should be different from that used to check the correct MS alignment, be commutable and with concentration corresponding to the decision cut-point employed in the medical application of the test [7], [9]: none of these features was met in the mentioned study.

Importantly, the analytical performance of laboratory tests, to be fit-for-purpose, should meet performance specifications (PS) defined according to objective models [7]. Since outcome-based PS for MU of SARS-CoV-2 serologic tests are not available, considering the state of the art of the measurement performance could represent an acceptable interim approach. However, to derive PS for MU by using this model it is necessary to identify the highest quality of performance that is currently achievable and, therefore, the assessment of MU of commercially available MSs is needed to identify the MU from the best performing system as the desirable PS. To our knowledge, this is the first study evaluating the random MU of a SARS-CoV-2 immunoassay employed routinely in a medical laboratory and no other studies have provided similar information. The state of the art of MU for anti-SARS-CoV-2 assays is therefore still unknown.

While various studies have been performed to assess and validate the clinical performance of different serologic tests, and the need for better evaluation criteria for clinical accuracy has been postulated [10], discussion about analytical quality of these tests, and relative acceptance criteria, has been scarce if not fully absent. In our opinion, this represents a serious drawback as high levels of MU may have a detrimental impact on clinical performance of the test causing patient misclassification. While the presence of a ‘grey-zone’ in result interpretation (at least for the CE-marked DiaSorin assay) may provide some protection against false-negative results, requiring a retest of patients with IgG titles between 12 and 15 AU/mL, for false-positive results no such safety net exists and, therefore, lower MU at cut-off levels becomes of extreme importance for accurately defining the immune status of patients suspected of having SARS-CoV-2 infection.

In the last months, we have often heard the argument that any testing for COVID-19 is better than none, and we have seen tender notices for which the only decision factor was the cost per test. As laboratory professionals, we can only respond that, as always in laboratory medicine, even for anti-SARS-CoV-2 serology bad (even if cheaper) assays will always be clinically (and socially) counterproductive. The poor analytical quality may be the bane of medical use of tests and even in the setting of serologic testing, the fight against the analytical variability presents a daily struggle. We must improve our knowledge by ensuring that serologic tests for SARS-CoV-2 antibodies perform as well as needed.