Comparison of the diagnostic performance with whole blood and plasma of four rapid antibody tests for SARS-CoV-2

To the Editor,

In the first quarter of 2020 more than 150 so called “rapid tests” for IgM/IgG antibodies against SARS-CoV-2, the cause of coronavirus disease 2019 (COVID-19), have come to the world market. The Food and Drug Administration (FDA) has issued emergency use authorizations for new lateral flow assays (LFA). In the EU, conformity for new devices for use by health professionals is self-declared by the manufacturer who has to specify device performance characteristics, with retrograde validation by national authorities [1].

The robustness and low cost make LFA particularly attractive in resource-limited settings, but they are also used in high resource countries [2]. These advantages are, however, balanced by fundamental challenges to the development of the assay, which include the lack of the independent control of primary and secondary binding reactions, loss of assay components due to incomplete binding, potential for false negatives due to the hook effect or false positives due to non-specific binding. These challenges are the reason that the quality of LFA can vary considerably although the same antigen is used [3].

Rapid detection of antibodies against SARS-CoV-2 using LFA could provide important opportunities and improve the diagnostic arsenal for the detection of SARS-CoV-2 infection by allowing point-of-care antibody testing without the need for centrifugation. There are, however, concerns regarding the performance of antibody LFA for the diagnosis of COVID-19. In the beginning of April, the British government reported problems with the performance of LFA with “many false negatives” and also “false positives” [4]. As a result of these problems, doctors and regulators throughout the world started to look with suspicion at rapid tests for COVID-19.

Despite the fact that most manufacturers claim that both whole blood and serum/plasma can be used, there is only a single study with a home-made LFA and only seven patients that compared whole blood and serum/plasma [5]. Validation studies for antibody LFA in general are typically performed using stored serum/plasma since it is difficult to obtain whole blood. If LFA are to be used at the point of care, however, it is essential that their performance with whole blood is comparable to serum/plasma.

We compared the diagnostic performance using whole blood and plasma of 4 Conformité Européenne (CE)-IVD marked LFA for professional use only. The assays are: MultiG single lane (MultiG1, lot NCP-20030181), MultiG dual lane (MultiG2, lot COV1452003C), Orientgene (lot 2003318), and Surescreen (lot COV20030120). Sensitivity was evaluated using 33 left-over samples from 26 patients (15 men, 11 women) with COVID-19 who had a positive SARS-CoV-2 PCR result 21–62 days before the experiments were performed. Samples were collected 23–65 days after reported onset of symptoms, when seroconversion for IgG is expected to have occurred [6]. Median age was 67 years (33–92 years) old. The control group consisted of 39 patients without a clinical suspicion of COVID-19 and who had a negative result for PCR in the last seven days. The study was performed at the University Hospital Leuven and approved by the local Ethics Committee (protocol number S63897).

To allow direct comparison of the results, LFA were first tested using left-over venous whole blood samples (K₃-EDTA) that were less than 36 h old. Next, samples were centrifuged and the LFA was performed a second time using EDTA plasma. Results were read by two independent readers. There were no disagreements between the two readers. Categorical data were compared with Fisher’s exact test.

Table 1 shows the results for IgM alone, IgG alone and IgM/IgG (positive for IgM and/or IgG) with whole blood and plasma. Sensitivity and specificity of IgG in patients was good (≥94.0 and ≥97.4%, respectively) and the agreement between whole blood and plasma for all four LFA was ≥97.0%. The specificity for IgM in controls was ≥94.9%, but the sensitivity IgM varied significantly between 45.4 and 87.9% for whole blood (p<0.01) and between 27.3 and 90.9% for plasma (p<0.01). The agreement for IgM between whole blood and plasma varied between 75.8% for MultiG2 and 97.0% for Surescreen (p<0.05). IgM antibodies remained detectable up to 62 days after the positive PCR-result for two patients. There were 17 weak positive bands (intensity less than the control band) with the four assays combined for IgM with whole blood and 16 with plasma compared to zero weak positive bands with whole blood and two weak bands with plasma for IgG (p<0.01 for whole blood and plasma).

Our study is the first study that compared the diagnostic performance of commercially available LFA for SARS-CoV-2 with whole blood and serum/plasma. The concordance between venous whole blood and plasma was good for IgG (≥97.0%), but varied significantly for IgM. The good sensitivity of IgG LFA in COVID-19 patients and significant intra-assay variation for IgM LFA are in line with our previous results [7]. Our results suggest that LFA can be used at the point of care for detection of IgG anti-SARS-CoV-2 antibodies. We do, however, not recommend to include IgM in the interpretation as this decreases the specificity without significantly improving diagnostic performance for LFA [7].

There are a few limitations to our study. First, the validation was performed using EDTA venous whole blood which is not the same as capillary whole blood (product inserts just mentioned “whole blood”). While venous whole blood offers the advantage that no centrifugation is required and the test can therefore be performed at the point of care, it still requires venous blood sampling. The use of capillary blood, however, is more prone to preanalytical errors [8]. Second, the study population was relatively small. We do believe that the fact our study was performed using four different LFA with an agreement of ≥97% for IgG for the four LFA balances out this shortcoming.

In conclusion, the diagnostic performance of the four LFA with whole blood and plasma was comparable. The sensitivity for IgG 21–62 days after a positive PCR and specificity for IgG were ≥94%. The results for IgM varied significantly between the four LFA for whole blood and plasma and there were significantly more weak positive results for IgM than for IgG. IgM antibodies remained detectable up to 62 days after the positive PCR result.