Introduction

The 2019 coronavirus pandemic (COVID-19) continues to spread worldwide. The effective isolation and early treatment of patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) require rapid, accurate, and straightforward diagnostic tools.

While currently recommended nucleic acid amplification tests (NAAT), such as real-time reverse transcription-polymerase chain reaction (rtRT-PCR) assays, remain the gold standard cornerstone for the diagnosis of SARS‐CoV‐2 infection [1, 2], viral antigens can be detected using immunological methods [2,3,4]. Indeed, conducting rtRT-PCR is expensive, time-consuming, and requires special equipment and qualified operators. Point-of-care antigen-detecting rapid diagnostic tests (Ag-RDT) constitute simple and less expensive alternative tests [3]. Ag-RDT relies on direct detection of SARS-CoV-2 viral proteins in nasal swabs and other respiratory secretions. The N-nucleocapsid protein is frequently targeted because of its relative abundance and conserved structure, or other viral proteins such as the spike protein [4]. Most Ag-RDTs rely on sandwich catching using anti-SARS-CoV-2 monoclonal antibodies to detect viral antigens in the simple-to-use lateral flow immunoassay format allowing results in < 30 min. However, significant variability has been reported about their diagnostic performance and a lack of external validation for many available tests, which still require clinical validation [5,6,7,8,9].

Our study aimed to evaluate the qualitative membrane-based immunochromatographic BIOSYNEX COVID-19 Ag BSS Ag-RDT (Biosynex Swiss SA, Freiburg, Switzerland; reference SW40006; abbreviated by BIOSYNEX Ag-RDT) using monoclonal antibodies detecting SARS-CoV-2 N-nucleocapsid protein to diagnose COVID‐19 from prospectively collected nasopharyngeal secretion samples in adults living in the Paris region throughout the third wave of the COVID-19 epidemic in France.

Materials and methods

Rapid antigen test

The BIOSYNEX Ag-RDT consists of a reaction membrane and three buffers (sample, reagent, and absorbent). The reagent buffer contains colloidal gold particles conjugated with monoclonal antibodies directed against the N protein of SARS-CoV-2. Secondary antibodies against the N protein are fixed on the reaction membrane. The manufacturer’s instructions were followed to conduct the test by mixing nasopharyngeal secretions with 300 µl of dilution buffer in a tube. After 1 min, four drops were added to the well on the cassette.

If SARS-CoV-2 antigens are present in the sample, the complexes between the anti-SARS-CoV-2 conjugate and the virus are captured by anti-SARS-CoV-2 monoclonal antibodies specific to the test line area (T). The lack of the T line indicates that the result is negative. A red line appears in the control line area (C) to serve as a procedural control, indicating that the correct sample volume has been added and the membrane has played its role. Reading is carried out after 15 min.

Study population and procedures

During the third wave of the COVID-19 epidemic (March and April 2021), two sites had been used to consecutively collect paired nasopharyngeal swabs. Site A was the Centre Cardiologique du Nord, Saint-Denis, France. Site B was the Laboratoire Paris XV, Paris, France. Participants aged ≥ 18 years and those consenting to undergo two nasopharyngeal swabs for rtRT-PCR and Ag-RDT were included. All participants were given a questionnaire that recorded demographic information (sex and age), reasons for testing, and current and past 14-day symptoms in symptomatic patients. Suggestive symptoms of COVID-19 were headache, fatigue, fever, or upper or lower respiratory symptoms. Asymptomatic individuals were defined as those not reporting any of these symptoms. At both sites, a health care professional first collected nasopharyngeal secretions in one nostril, using the swab provided in the BIOSYNEX Ag-RDT. A second nasopharyngeal swab in the other nostril served as specimen for the rtRT-PCR. The COVID-19 antigen rapid test was performed immediately on-site using the Ag-RDT following the manufacturer’s instructions. The other nasopharyngeal swabs were stored in physiological saline (NaCl 0.9%) (1000 μL) at + 4 °C and analyzed within 24–48 h by the reference rtRT-PCR.

Molecular detection of SARS-CoV-2

The multiplex real-time PCR Novel Coronavirus (2019-nCoV) Real-Time Multiplex RT-PCR Kit (Detection for 3 Genes) (Liferiver & Shanghai ZJ Bio-Tech Co., Ltd, Shanghai, China) was the reference multiplex molecular detection of SARS-CoV-2 RNA. Individual cycle threshold (Ct) values for each target gene (E, N and RdRP). According to manufacturer’s recommendations, samples with Ct values ≤ 41 for three or two gene targets were considered as positive; those with Ct values ≤ 41 for only one gene target were possibly positive; samples with Ct value > 41 for the three gene targets were negative.

The Ct values of the N gene in the RT-PCR reference were chosen for stratification of viral load in clinical samples because the Ag-RDT detects the SARS-CoV-2 N-nucleocapsid protein.

Statistical analyses

Collected data were analyzed using IBM® SPSS® Statistics 20 software (IBM, SPSS Inc, Armonk, New York, USA). Results of quantitative variables were expressed as medians; however, the proportion with their 95% confidence interval (CI) assessed according to the Wilson score bounds were estimated for categorical variables [10]. Comparisons were carried out using Pearson’s Chi square test or Fisher’s exact test based on validity conditions. The PPV and NPV were calculated according to Bayes’ formulas, taking into account the officially reported prevalence of SARS-CoV-2 RNA positivity in symptomatic patients in the Paris region on 12th April 2021, e.g., around the peak of the third wave epidemic in France (Santé publique France 2021; https://www.santepubliquefrance.fr/).

Ethics statement

The purpose of the study was to clinically evaluate the continuous quality improvement program and performance evaluation of COVID-19 management measures following the National Medical-Biological Laboratory Accreditation [11]. The data set was anonymous and contained no identifiable personal health information.

Results

Paired swab samples were obtained from 967 participants, including 741 from site A and 226 from site B (Table 1). Participants ranged in age from 18 to 95 (median = 34 years). The main reasons for testing were air travel (35.6%), contact-case exposure of an individual infected with SARS-CoV-2 (35.1%), suspected COVID-19 (n = 212, 21.9%), preoperative assessment (4.4%), and control of SARS-CoV-2 infection in the previous 30 days (3.0%). The majority (722/967, 74.7%) of included persons were asymptomatic, while a minority (245/967, 25.3%) reported at least one COVID-19-related symptom [including 212 suspected COVID-19 cases, 29 (8.5%) contact cases, 3 (0.9%) travelers, and 1 (3.0%) patient with a recent history of COVID-19]. The median symptom duration before sampling was four days (range, 0–20 days). All comparisons between positive and negative Ag-RDT and rtRT-PCR testing results for both sites and all other variables did not achieve statistical significance (not shown).

Table 1 Characteristics of persons providing paired upper respiratory swab (n = 967) for real-time reverse transcription-polymerase chain reaction (rtRT-PCR) testing and BIOSYNEX COVID-19 Ag BSS rapid diagnostic testing for SARS-CoV-2 at two private laboratory sites, by test results, Paris, France, spring 2021
Full size table

Among the 148 positive samples using the gold standard rtRT-PCR, 146 were positive for the three gene targets, and two were positive for only E and N genes. The mean ± SD of the Ct values were 26.1 ± 4.4 arbitrary units (a.u.) for the E gene, 26.5 ± 5.0 a.u. for the RdRP gene, and 26.9 ± 5.1 a.u. for the N gene.

The vast majority (114/123, 92.7%) of positive results were visible in the window of the cassette of the Ag-RDT within the first 5 min. Table 2 shows the test results and primary performance characteristics of the BIOSYNEX Ag-RDT compared with the reference rtRT‐PCR in the study population according to COVID-19-compatible symptoms. Using rtRT-PCR as the standard, three false-positive BIOSYNEX Ag-RDT results occurred among specimens from asymptomatic individuals (n = 2) or symptomatic patients (n = 1). Of the 148 rtRT-PCR positive results, 27 (18.2%) were false-negative BIOSYNEX Ag-RDT (23 specimens from asymptomatic persons and 4 specimens from symptomatic patients). Overall, the BIOSYNEX Ag-RDT showed high sensitivity (81.8%), specificity (99.6%), PPV (96.6%), and NPV (97.5%). Among symptomatic patients, sensitivity was 95.0%, specificity was 99.4%, PPV was 95.6%, and NPV was 96.3% (Table 2). Within 7 days from symptom beginning, the BIOSYNEX Ag-RDT showed a sensitivity of 96.6%, a specificity of 99.4%, whereas the PPV and NPV were 95.7% and 99.4%, respectively.

Table 2 Test results and performances characteristics of the BIOSYNEX COVID-19 Ag BSS rapid diagnostic test compared with real-time reverse transcription-polymerase chain reaction (rtRT-PCR) for SARS-CoV-2 testing among asymptomatic and symptomatic persons at two private laboratory sites, by test results, Paris, France, spring 2021
Full size table

The Table 3 shows the analytical results based on the level of viral excretion assessed by the N gene Ct values using the reference rtRT‐PCR. Overall, the BIOSYNEX Ag-RDT showed high agreement (97.0%), reliability using Cohen’s κ coefficient (0.87), and accuracy using Youden’s J index (81.6%) to detect SARS-CoV-2.

Table 3 Analytical performances of the BIOSYNEX COVID-19 Ag BSS rapid diagnostic test for the qualitative detection of the N protein of SARS-CoV-2 using 967 prospectively collected nasopharyngeal swab samples by reference rtRT-PCR#, according to their N gene Ct values
Full size table

In case of high or very high viral loads (Ct ≤ 33), the BIOSYNEX Ag-RDT had a good analytical performance (sensitivities between 83.3% and 100.0%, specificities of 99.8%, PPV between 98.3% and 98.6%, and NPV between 97.7% and 100.0%). In case of low or very low viral loads (Ct > 33), the sensitivity of the BIOSYNEX Ag-RDT had reduced analytical performance (sensitivity of only 55.2%), while its specificity remained high (98.8%). Similar observations were made when the Ct values of the E or ORF1ab gene targets were chosen for stratification of viral load in clinical samples (data not shown).

Finally, the sensitivity of the BIOSYNEX Ag-RDT varied among the five participant groups as follows: (i) travel: 50.0% (7/14), (ii) contact-case exposure: 81.0% (47/58), (iii) preoperative assessment: 50.0% (2/4); (iv) suspected COVID − 19: 96.8% (61/63), and (v) control of SARS-CoV-2 positive test results in the last 30 days: 88.9% (8/9).

Discussion

We evaluated the analytical performance of the novel point-of-care BIOSYNEX Ag-RDT compared to multiplex rtRT-PCR as gold standard for detecting SARS-CoV-2 RNA in a real-life setting. In this study, the sensitivity of the BIOSYNEX Ag-RDT was lower among specimens from asymptomatic persons (79.4%) than among specimens from symptomatic patients (95.0%). It was high in patients with suspected COVID-19 (96.8%). Specificity (> 99.0%) was high in specimens from both asymptomatic individuals and symptomatic patients. The prevalence of SARS-CoV-2 RNA-positive rt-RT-PCR results in this population was 15.3% overall, 9.4% for asymptomatic individuals, and 32.6% for symptomatic patients. The estimated PPVs and NPVs of the BIOSYNEX Ag-RDT were elevated in all groups of participants. However, administering the Ag-RDT in low prevalence settings will likely result in lower predictive values. In the event of significant viral excretion (i.e., N gene Ct values below 33 based on reference rtRT-PCR), the BIOSYNEX Ag-RDT showed high sensitivity (from 83.3% to 100.0%) and specificity (> 99.0%) for SARS-CoV-2 RNA detection. Concordance, reliability, as well as accuracy were great with the reference assay and PPVs and NPVs above 97.0%. However, the sensitivity of the study Ag-RDT dropped to 55.2% with low or very low viral shedding (Ct > 33). Together, these observations demonstrated the high analytical performance of the BIOSYNEX Ag-RDT. This performance made it suitable for use as point-of-care Ag-RDT in various hospital and non-hospital settings where a rapid diagnosis of SARS-CoV-2 is necessary. Although less sensitive than RT-PCR, the BIOSYNEX Ag-RDT could be beneficial by obtaining quick results, ease of use, and independence from existing laboratory structures. Testing criteria focusing on patients during the early onset of symptoms could further increase its diagnostic value.

The sensitivity of the BIOSYNEX Ag-RDT was 81.8% overall, and the positive detection rate was comparable to the rtRT-PCR in the majority (88.2%) of subjects with Ct ≤ 33. False-negative test results of 12/14 (85.7%) subjects with significant viral excretion (Ct ≤ 33) were asymptomatic, although conflicting evidence exists about the relationship between symptom severity and viral shedding [12]. False-positive test results were rarely observed, providing 99.6%-specificity, exceeding the performance recommended by the World Health Organization (WHO) [13]. False-positive results have been reported as well in other antigen tests [14,15,16]. False positivity could be associated with high viscosity of tested specimen samples as well as interference with mucosal antibodies [17].

Finally, the BIOSYNEX Ag-RDT meets the current WHO criteria which stipulate that Ag-RDTs for SARS-CoV-2 antigen detection must have a sensitivity greater than 80% and a specificity greater than 97% (97%–100%) [13]. Furthermore, analytical performances comparable to those in our study Ag-RDT were previously reported for some Ag-RDTs in lateral flow immunoassay format [7, 9, 14, 18,19,20,21,22,23,24,25,26,27,28], while several studies have reported much lower sensitivity levels contrasting with consistently high specificity [3, 29,30,31,32,33,34]. In addition, the BIOSYNEX Ag-RDT also fulfilled the current recommendations of the French High Authority of Health (Haute Autorité de santé, Saint-Denis, France) for a screening Ag-RTD stating that, at minimum, Ag-RDTs would need to correctly identify significant proportions of symptomatic patients (sensitivity ≥ 80%) as well as asymptomatic individuals (sensitivity ≥ 50%) and have high specificity (≥ 90%) [35].

We analyzed our results based on the estimated viral load in SARS-CoV-2 in the samples. There is an ongoing debate about the Ct value corresponding to the threshold of infectivity (i.e., patient considered as contagious) [7, 36, 37]. La Scola et al. found that patients with Ct values > 33 are not infectious because of the low number of positive cultures [38]. The Centers for Disease Control and Prevention (CDC), Atlanta, USA, propose a Ct cut-off value of 33 as a marker for contagiousness [39], and stress that Ct values ≤ 20 correspond to very high viral excretion [7, 36, 40, 41]. Our results confirm that the analytical performances of the BIOSYNEX Ag-RDT were much better in specimens with a high viral load. These observations demonstrate the capability of the BIOSYNEX Ag-RDT as a rapid rule-in test for COVID-19 with samples at high viral load in symptomatic patients, for example, and raise caution about its use as a singular rule-out test, especially in samples with lower viral loads.

Our study has several strengths. All samples were collected from one nasopharynx with flocked swabs, optimal for evaluating Ag-RDT clinical performances in our study. The Ag-RDT and reference rtRT-PCR were carried out in parallel. The study population included various situations outside the hospital setting, with mostly young adults without comorbidities who had typical and mild COVID-19 symptoms when being symptomatic.

The study presents also some limitations. Participants may have inadvertently reported general, non-specific symptoms as COVID-19 compatible symptoms. This investigation evaluated the BIOSYNEX Ag-RDT; the results presented here cannot be generalized to other agencies-authorized SARS-CoV-2 antigen tests. Otherwise, the CDC clarified that Ct values using the rtRT-PCR platform is not a quantitative measure of viral burden in clinical samples and cannot be used to assess whether a person is infectious [42]. Consequently, our stratification of samples according to Ct values of the N gene does not necessarily reflect the actual infectivity of the participants. Finally, higher rate of asymptomatic persons in the study (with lower virus level) could have resulted in decreased sensitivity of the BIOSYNEX Ag-RDT.

Conclusion

The BIOSYNEX Ag-RDT demonstrated high specificity and sufficient sensitivity for detecting SARS-CoV-2. Given the simple procedures and short turnaround time for this test, it is a promising option as an alternative diagnostic modality, especially in symptomatic COVID-19 patients. The test may also be used to test asymptomatic individuals suspicious of exposure to SARS-CoV-2 and as part of a population-level mass screening.