Next Article in Journal
An Artificial Placenta Experimental System in Sheep: Critical Issues for Successful Transition and Survival up to One Week
Next Article in Special Issue
Clinical Performance of Lateral Flow Assay for Cryptosporidium spp. Diagnosis
Previous Article in Journal
The Proteomes of Oral Cells Change during Co-Cultivation with Aggregatibacter actinomycetemcomitans and Eikenella corrodens
Previous Article in Special Issue
Clinical Validation of GenBody COVID-19 Ag, Nasal and Nasopharyngeal Rapid Antigen Tests for Detection of SARS-CoV-2 in European Adult Population
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of Four Rapid Antigen Tests for the Detection of SARS-CoV-2 Infection with Nasopharyngeal Swabs

1
Department of Molecular Diagnostics, Seegene Medical Foundation, Seoul 04805, Republic of Korea
2
Department of Integrative Biological Sciences & BK21 FOUR Educational Research Group for Age-Associated Disorder Control Technology, Chosun University, Gwangju 61452, Republic of Korea
3
Paul F. Glenn Center for Biology of Aging Research, Department of Genetics, Blavatnik Institute, Harvard Medical School, Boston, MA 02115, USA
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Biomedicines 2023, 11(3), 701; https://doi.org/10.3390/biomedicines11030701
Submission received: 17 January 2023 / Revised: 16 February 2023 / Accepted: 23 February 2023 / Published: 24 February 2023

Abstract

:
Owing to the high transmissibility of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) variants, the capacity of testing systems based on the gold standard real-time reverse transcription–polymerase chain reaction (rRT-PCR) is limited. Rapid antigen tests (RATs) can substantially contribute to the prevention of community transmission, but their further assessment is required. Here, using 1503 nasopharyngeal swabs, we compared the diagnostic performance of four RAT kits (Abbott Panbio™ COVID-19 Ag Rapid Test, SD Biosensor Standard™ Q COVID-19 Ag Test, Humasis COVID-19 Ag Test, and SG Medical Acrosis COVID-19 Ag Test) to the cycle threshold (Ct) values obtained from rRT-PCR. The precision values, area under the curve values, SARS-CoV-2 variant detection ability, and non-SARS-CoV-2 specificity of all four kits were similar. An assay using the Acrosis kit had a significantly better positive detection rate with a higher recall value and cut-off value than that using the other three RAT kits. During the current COVID-19 pandemic, the Acrosis kit is an effective tool to prevent the spread of SARS-CoV-2 in communities.

1. Introduction

The coronavirus disease 2019 (COVID-19) pandemic, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), started in December 2019, and it remains a global public health concern [1]. The COVID-19 pandemic has subsided to a seasonal illness through vaccination campaigns [2]. Despite stringent efforts to control the spread of SARS-CoV-2, more than 633 million confirmed cases and 6.6 million deaths had been reported as of 17 November 2022 [3]. The fight against SARS-CoV-2 is hampered by coinfection with other respiratory viruses, such as influenza virus, respiratory syncytial virus, and non-SARS-CoV-2 coronavirus [4].
SARS-CoV-2 has one of the largest RNA genomes (~29.9 kb) among all RNA viruses [5]. Owing to the high mutation rate of RNA viruses, SARS-CoV-2 continuously accumulates multiple mutations, resulting in numerous variants of concern by December 2020 [6]. These variants have shown increased transmissibility because of extensive mutations and available diagnostic tools have become less effective [7]. Currently, various diagnostic tests for SARS-CoV-2 include real-time reverse transcription–polymerase chain reaction (rRT-PCR)-based nucleic acid assays, immunoassays (e.g., lateral flow antigen test), and clinical imaging (e.g., computed tomography scan) [8].
The rRT-PCR assay is the gold-standard test for SARS-CoV-2 detection, especially when using nasopharyngeal swabs (NPSs) obtained from suspected cases by trained healthcare workers [9]. However, rRT-PCR assays have a turnaround time of approximately 24–48 h, which results in a bottleneck in PCR testing during the peak period of SARS-CoV-2 infections, which adversely affects the management of patients with COVID-19 [10]. Thus, there is a need for further studies to develop and evaluate more rapid and feasible assays, such as rapid lateral flow tests.
Unlike rRT-PCR, the rapid lateral flow test employs direct detection methods, is easy to perform (and does not need to be performed by healthcare workers), and yields results within 30 min [11]. However, rapid lateral flow tests for the detection of antigens are considered less sensitive and less specific than rRT-PCR [12]. To overcome these limitations, various signal-amplified methods have been used for the sensitive detection of antigens in rapid lateral flow tests, including chemical enhancement, surface-enhanced Raman scattering, and fluorescence, as well as photothermal, electrochemical, and magnetic reactions [13]. Numerous studies have evaluated the sensitivity and reliability of the commercially available rapid antigen test (RAT) kits for SARS-CoV-2 detection [11,14,15]. However, only a few studies have analyzed the specificity of RATs for other respiratory viruses and SARS-CoV-2 variants or compared the results with the cycle threshold (Ct) values obtained from rRT-PCR assays [12,16].
To date, studies have demonstrated the performance of well-known rRT-PCR kits for SARS-CoV-2 detection [17,18]. Therefore, in this study, we compared the diagnostic performance of four RAT kits, namely the ACROSIS COVID-19 Antigen kit (SG Medical, Seoul, Republic of Korea), STANDARDTM Q COVID-19 Ag Test kit (SD BIOSENSOR Inc., Suwon, Republic of Korea), PanbioTM COVID-19 Ag Rapid Test kit (Abbott Diagnostics, Chicago, IL, USA), and Humasis COVID-19 Ag Test kit (Humasis Co., Ltd., Gunpo, Republic of Korea) to the Ct values obtained from rRT-PCR. These RAT kits are considered to yield results more quickly than the rRT-PCR assay with comparable accuracy. Therefore, we compared the diagnostic performance of these RAT kits with that of rRT-PCR using large-scale NPS samples (≥1500).

2. Materials and Methods

2.1. Storage of Clinical Specimens

This study was approved by the Institutional Review Board of the Seegene Medical Foundation (SMF-IRB-2022-025 approved on 25 July 2022). The requirement for informed consent from the participants was waived because the samples collected for this study were anonymized. NPS specimens (n = 1503) were obtained during routine diagnostic testing conducted from December 2020 to October 2022 and evaluated using the SARS-CoV-2 molecular assay. The results of the analysis of cross-reactivity tests showed that 67 of the SARS-CoV-2-negative (n = 701) samples provided were positive for other known respiratory viruses. Briefly, the samples with Ct values < 26 were considered positive and classified under 14 species as follows: human coronaviruses OC43 (n = 5), NL63 (n = 4), and 229E (n = 3); influenza virus types A (n = 5) and B (n = 5); respiratory syncytial virus types A (n = 5) and B (n = 6); parainfluenza virus types 1 (n = 4) and 4 (n = 5); bocavirus (n = 5); enterovirus (n = 5); rhinovirus (n = 5); metapneumovirus (n = 5); and adenovirus (n = 5). The specimens were anonymized, aliquoted into 1.5 mL conical tubes, and stored at −70 °C until use (freeze/thaw cycles ≤ 1). Samples of sufficient volume (≥700 µL) for all four RATs, as well as for nucleic acid extraction before rRT-PCR, were preserved in a universal transport medium (UTM) without inactivation.

2.2. Nucleic Acid Extraction and rRT-PCR Analysis

The rRT-PCR assay was used as the gold standard; the assay was performed with purified nucleic acid and a commercially available kit using the AllplexTM SARS-CoV-2 Assay (Seegene Inc., Seoul, Republic of Korea). The nucleic acid was extracted from 1503 post-thaw samples using the MagNA Pure 96 system (Roche Inc., Basel, Switzerland) as previously described [19]. The purified nucleic acids were then used in the rRT-PCR, which detects the RNA-dependent RNA polymerase (RdRP) and spike (S), envelope (E), and nucleocapsid (N) genes. The rRT-PCR assay with the AllplexTM SARS-CoV-2 Assay kit was performed using CFX96TM (Bio-Rad Laboratories, Hercules, CA, USA). The thermal cycling conditions were as follows: 50 °C for 20 min; 95 °C for 15 min; and 45 cycles at 95 °C for 10 s, 60 °C for 15 s, and 72 °C for 10 s. The data were analyzed using Seegene Viewer (v3.24; Seegene Inc.). The results were interpreted as SARS-CoV-2 positive when the Ct value was ≤40 for all targets, according to the manufacturer’s instructions [9]. Of these, RdRP and S were represented by a Ct value from rRT-PCR in this study. Positive samples were then categorized into three different groups based on their Ct value: Ct < 20, 20 ≤ Ct < 30, and Ct ≥ 30. Of these, the 20 ≤ Ct < 30 group was subdivided into five groups with Ct values of 20 ≤ Ct < 22, 22 ≤ Ct < 24, 24 ≤ Ct < 26, 26 ≤ Ct < 28, and 28 ≤ Ct < 30.

2.3. RAT Kits and Experimental Protocol

The 1503 thawed samples were tested simultaneously for the SARS-CoV-2 N protein using four commercially available RAT kits: (i) PanbioTM COVID-19 Ag Rapid Test kit (PanbioTM), (ii) STANDARDTM Q COVID-19 Ag Test kit (StandardTM), (iii) Humasis COVID-19 Ag Test kit (Humasis), and (iv) Acrosis COVID-19 Antigen kit (Acrosis). The standard procedure was modified according to a previously reported protocol [20,21]. Briefly, we mixed 60 μL of UTM with 60 μL of extraction buffer (1:1 ratio); this is not routinely applied in RAT protocols. The mixtures were transferred to their respective sample wells following the manufacturer’s recommendation and were maintained at 20–24 °C for an additional 15 min. The results were then analyzed by two authors (Y.H.B. and H.S.L.); a positive result was one in which a band was visible at the “T” site. In the event of discrepant interpretations, another author was consulted to make a final decision (H.J.L. or Y.Y.).

2.4. Next Generation Sequencing (NGS)

Using the RAT kit-tested samples, an NGS analysis was conducted below the minimum cut-off of the randomly selected SARS-CoV-2-positive samples using the MiSeq platform (Illumina, San Diego, CA, USA) following a previously published protocol [22]. Briefly, 55 ng of nucleic acid were processed using SARS-CoV-2 FLEX Panels (Paragon Genomics, Hayward, CA, USA). The sequence files were analyzed using the Flomics pipeline (Flomics, Barcelona, Spain) and were assigned to a sublineage by the Pangolin Tool (ver.4.1.3). Genetic mutations in N were identified by Nextclade (ver. 2.8.1).

2.5. Statistical Analysis

Rstudio (ver. 4.1.2; RStudio Inc., Boston, MA, USA) was used to perform all statistical analyses and generate graphs. The performance of each RAT kit was evaluated by estimating precision and recall against the rRT-PCR results using the caret package [23]. The correlation between the Ct values from rRT-PCR and the results of assays using each RAT kit was evaluated using the log-rank test in the survminer package [24]. The receiver operating characteristic (ROC) curve and area under curve (AUC) were generated using the pROC package, and the optimal cutoffs for each RAT were calculated using scores that estimated the minimum distance between the left-upper corner and ROC curve [25]. The dot plots were generated using the ggpubr package. The true-positive rate (proportion of RAT-positive results in SARS-CoV-2-positive samples) was used to compare the differences among the RAT kits. The results were considered statistically significant at p < 0.05.

3. Results

3.1. Comparison of Detection Rates of the RAT Kits

The clinical performance of the four RAT kits was compared using 1503 samples. Of the 1503 NPS samples, 802 were SARS-CoV-2 positive, as determined using rRT-PCR (Table 1). Without inhibiting the control line, all the RAT kits were effective for the UTM samples. Compared with the rRT-PCR assay, the assay using the RAT kits had a precision rate of 99.32–100%; however, the recall rate was 76.56% for Acrosis and between 53.87% and 55.74% for the other three kits (Table 1). Overall, the performance of the Acrosis kit was better than that of the other three kits for the SARS-CoV-2-positive samples.

3.2. Positivity and Cross-Reactivity of the RAT Kits

The 802 SARS-CoV-2-positive samples in the rRT-PCR analysis (Table 2) were classified based on their Ct value (Ct < 20: n = 137; 20 ≤ Ct < 30: n = 513; or 30 ≤ Ct < 40: n = 152). Of these, 513 samples (20 ≤ Ct < 30) were further classified as follows: 20 ≤ Ct < 22 (n = 110), 22 ≤ Ct < 24 (n = 96), 24 ≤ Ct < 26 (n = 100), 26 ≤ Ct < 28 (n = 102), and 28 ≤ Ct < 30 (n = 105).
The RAT results showed that 29.5% (444/1503), 29.8% (448/1503), 28.7% (432/1503), and 40.9% (614/1503) of the samples tested positive in the assays using the PanbioTM, StandardTM, Humasis, and Acrosis kits, respectively. Of the 701 samples that tested negative in the rRT-PCR analysis, three were positive (0.4%) in the assay using the PanbioTM kit and one (0.1%) was positive in the assay using the StandardTM kit (Table 2). We compared the cross-reactivity of RATs using samples infected with other respiratory viruses in the SARS-CoV-2-negative group. The cross-reactivity of 67 samples was 100% with no false positives for SARS-CoV-2 (Supplementary Table S1). Of the remaining 802 SARS-CoV-2-positive samples, two RAT kits (PanbioTM and Humasis) reliably detected samples with a Ct value of ≤20 per kit. However, the other two RAT kits, StandardTM and Acrosis, detected samples with values of 22 and 24 Ct, respectively. The Acrosis kit was reasonably sensitive, showing 100% SARS-CoV-2 detection in samples with a Ct value < 24 (Table 2). In addition, it showed no cross-reactivity for 14 species of non-SARS-CoV-2 pathogens (Supplementary Table S1). The log-rank test was used to compare the positive rate in the assays with the four kits (Table 3). There was no significant difference in the results obtained using the StandardTM, PanbioTM, and Humasis kits (p > 0.05). However, there was a significant difference in the positive rate in the assay using the Acrosis kit compared with the other three RAT kits (p < 0.001).

3.3. Optimal Cut-Off Values Based on Ct Values

The ROC curves were constructed to examine the diagnostic accuracy of the four RAT kits (Figure 1). The cut-off scores of the RAT kits are summarized using dot plots (Supplementary Figure S1). The AUCs of the Ct values, which were used to discriminate between RAT-positive and RAT-negative results, ranged from 0.947 to 0.966. The highest optimal cut-off score determined from the Acrosis kit was 28.57, whereas the cut-off scores for the other three kits ranged from 25.58 to 25.79. Thus, the cut-off value for the Acrosis kit was higher (by 2.78–2.99) than that for the other three kits.

3.4. True-Positive Rate of the RAT Kits for SARS-CoV-2 Variants below the Cut-Off Values

Among the 427 SARS-CoV-2-positive samples with Ct values below 25.58, 77 (18.0%) were analyzed using NGS (Table 4). The NGS analysis confirmed the three SARS-CoV-2 variants (pre-Delta, Delta, and Omicron). The SARS-CoV-2 variants were classified into 10 lineages as follows: 3 lineages of pre-Delta variants (8 samples for B.1.497, 3 for B.1.619, and 4 for B.1.620), 2 lineages of Delta variants (3 samples for AY.69 and 4 for AY.122.5), and 5 lineages of Omicron variants (2 samples for BA.1, 17 for BA.2, 1 for BA.4, 33 for BA.5, and 2 for BE.1). Of these, three to five samples had discrepant results in the analysis using the PanbioTM kit (two samples for B.1.620 and two for BA.5), StandardTM kit (one sample for B.1.620 and two for BA.5), and Humasis kit (two samples for B.1.620 and three for BA.5). These findings suggest that the Acrosis kit has a remarkable superiority in detecting the 10 lineages of the three SARS-CoV-2 variants. For social activity settings requiring a negative SARS-CoV-2 test, the Acrosis kit can serve as an effective tool for regular screening.

3.5. Additional Mutations in Samples of the B.1.619 and BA.5 Lineages with Discordant Results

To determine the genetic mutation site causing discordant results in the RAT kit assays, an NGS analysis was performed. A220F was a common mutation in the B.1.620 lineage, and P13L, ERS31–33del, R203K, G204R, and S413R were identified as common mutations in the BA.5 lineage (Table 5). In the discordant samples analyzed using the RAT kits (Table 4), L221F was observed to be the additional mutation in the B.1.620 lineage in two samples, whereas the BA.5 lineage had no additional mutation in three samples (Table 5). There were no differences in additional mutations between the concordant and discordant samples. These findings suggest that the discordant samples would not have affected N mutations.

4. Discussion

SARS-CoV-2 is the third most highly pathogenic human coronavirus next to SARS-CoV and MERS-CoV and is rapidly transmitted from infected individuals [26]. SARS-CoV-2 accumulates genetic mutations, resulting in the emergence of new variants, such as pre-Delta (e.g., B.1.1.7, B.1.351, B.1.497, and B.1.619), Delta (e.g., B.1.617.2, AY.69, and AY.122.5), and Omicron variants (e.g., BA.1, BA.2, and BA.3) [27]. Newly emerging variants are rapidly evolving, thereby increasing both the number of confirmed cases and new mutations, which can affect the sensitivity of diagnostic tests [28]. Therefore, a rapid and accurate diagnostic test for SARS-CoV-2 infection is crucial for the effective prevention of disease transmission.
The RAT kits are simple and reliable on-site testing tools used for the rapid (less than 30 min) detection of SARS-CoV-2 infection from self-collected nasal swabs [11]. A previous study reported a similar performance between self-collected and healthcare-worker-collected samples [9]. The self-collection of nasal and oropharyngeal swabs is promoted to screen for widespread infection without delay [29,30]. It was an effective approach during the COVID-19 pandemic as it reduced the community transmission of SARS-CoV-2 [9]. Despite these advantages, numerous studies have reported a lower analytical sensitivity of RAT kits than that of rRT-PCR methods and analyzed a wide range of Ct values in the optimal cut-off score [31,32,33]. Thus, in this study, the cut-off values from Ct values between 20 and 30 were estimated using a similar number of samples.
The aim of this comparative study was to evaluate the performance of four commercial RAT kits to detect SARS-CoV-2 infection in NPS samples using the established rRT-PCR as a reference test. None of the RATs displayed cross-reactivity with 14 other respiratory viruses (Table S1), and all the tests had similar AUC values (Figure 1). However, three RAT kits (PanbioTM, StandardTM, and Humasis) showed divergent results for two lineages (among five samples) unlike the Acrosis kit (Table 4). The N mutations (P13L, ERS31–33del, A182T, R203K, G204R, A220F, L221F, and S413R) found in this study did not correlate between the consistent and inconsistent samples (Table 5). These findings indicate that these RATs are highly specific for SARS-CoV-2.
It has been reported that the screening of asymptomatic patients using RAT kits is limited [34] and the sensitivity of RAT kits reduces the number of true-positive results with Ct values > 25 [35,36]. The primary finding of this study was that the Acrosis kit showed a high recall rate (Table 1) and highly sensitive optimal cut-off Ct value of 28.57 (Supplementary Figure S1D) compared with the other three RAT kits (Supplementary Figure S1A–C). In terms of rRT-PCR Ct values, the performance of the Acrosis kit was significantly better than that of the other three kits (p < 0.001; Table 3). Notably, the Acrosis kit showed 6.87–7.94 fold (3 Ct) more sensitivity than the other three kits (Figure 1). Therefore, the Acrosis kit can be an effective screening tool for detecting infected individuals and protecting the community.
This study has two major limitations. First, the test kit manufacturers recommended the use of NPS specimens; these swabs should be placed in the provided extraction buffer. However, in this study, the UTM was mixed with the provided extraction buffer (at a ratio of 1:1). Second, the discrepancies identified in five samples could not be understood. Furthermore, the NPS samples were freeze thawed under identical sample conditions; however, the effect of the freeze-thawed samples on RATs has not been evaluated. Therefore, further studies to evaluate the RAT kits using fresh, unfrozen nasal swabs for the detection of SARS-CoV-2 infection are required.

5. Conclusions

Owing to the emergence of multiple, highly transmissible SARS-CoV-2 variants since December 2020, alternative testing methods with short turn-around times are required to prevent community virus transmission. In this study, we obtained three major findings. First, the true-positive rate of the Acrosis kit was significantly higher than that of the other three kits. Second, all RAT kits evaluated in this study can detect 10 lineages of SARS-CoV-2 variants (including Pre-Delta, Delta, and Omicron), despite differences in the cut-off value. Third, none of the RAT kits detected 14 other respiratory pathogens, indicating high specificity to SARS-CoV-2. Our results indicate that the Acrosis kit can be used in daily clinical practice for SARS-CoV-2 detection; however, additional studies on the field performance of the kit are required to determine its efficiency.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/biomedicines11030701/s1: Supplementary Table S1: Cross-reactivity of the RAT kits with other respiratory pathogens; Supplementary Figure S1: SARS-CoV-2-positivity rate according to the RAT kits (based on the Ct values).

Author Contributions

Conceptualization, M.-Y.P. and Y.-J.Y.; methodology, H.-J.L., M.-Y.P. and Y.-J.Y.; validation, H.-J.L., M.-Y.P., J.-E.P. and Y.-J.Y.; formal analysis, H.-J.L. and M.-Y.P.; investigation, H.-J.L., Y.-H.B. and H.-S.L.; data curation, Y.-H.B., H.-S.L., I.K., Y.K., Y.Y., K.N. and N.Y.; writing—original draft preparation, H.-J.L. and J.-E.P.; writing—review and editing, M.-Y.P., Y.-H.B., J.-H.Y., M.-J.K., Y.-H.S., J.-E.P. and Y.-J.Y.; visualization, H.-J.L., Y.-H.B. and J.-E.P.; supervision, M.-Y.P. and Y.-J.Y.; project administration, Y.-H.S. and Y.-J.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted in accordance with the guidelines of the Declaration of Helsinki and was approved by the Institutional Review Board of the Seegene Medical Foundation (SMF-IRB-2022-025 approved on 25 July 2022).

Informed Consent Statement

Patient consent was waived as per Article 16 Clause 3 of the Bioethics and Safety Act. Obtaining written consent from research participants was waived after approval from the competent Institutional Review Board that has authority over the presented cases. The legal compliance of this study was reviewed and approved by the Institutional Review Board of the Seegene Medical Foundation.

Data Availability Statement

All data are available within the article.

Acknowledgments

We thank SG Medical who kindly donated the Acrosis COVID-19 Antigen kit used in this study. Any draft, findings, and conclusions in this study do not reflect the views or positions of the manufacturer.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Rabi, F.A.; Al Zoubi, M.S.; Kasasbeh, G.A.; Salameh, D.M.; Al-Nasser, A.D. SARS-CoV-2 and Coronavirus Disease 2019: What We Know So Far. Pathogens 2020, 9, 231. [Google Scholar] [CrossRef] [PubMed]
  2. Byun, W.S.; Heo, S.W.; Jo, G.; Kim, J.W.; Kim, S.; Lee, S.; Park, H.E.; Baek, J.H. Is coronavirus disease (COVID-19) seasonal? A critical analysis of empirical and epidemiological studies at global and local scales. Environ. Res. 2021, 196, 110972. [Google Scholar] [CrossRef] [PubMed]
  3. World Health Organization. WHO Coronavirus (COVID-19) Dashboard. Available online: https://covid19.who.int (accessed on 17 November 2022).
  4. Pinky, L.; Dobrovolny, H.M. SARS-CoV-2 coinfections: Could influenza and the common cold be beneficial? J. Med. Virol. 2020, 92, 2623–2630. [Google Scholar] [CrossRef] [PubMed]
  5. Naqvi, A.A.T.; Fatima, K.; Mohammad, T.; Fatima, U.; Singh, I.K.; Singh, A.; Atif, S.M.; Hariprasad, G.; Hasan, G.M.; Hassan, M.I. Insights into SARS-CoV-2 genome, structure, evolution, pathogenesis and therapies: Structural genomics approach. Biochim. Biophys. Acta Mol. Basis. Dis. 2020, 1866, 165878. [Google Scholar] [CrossRef]
  6. Lauring, A.S.; Hodcroft, E.B. Genetic Variants of SARS-CoV-2—What Do They Mean? JAMA 2021, 325, 529–531. [Google Scholar] [CrossRef]
  7. Boehm, E.; Kronig, I.; Neher, R.A.; Eckerle, I.; Vetter, P.; Kaiser, L.; Geneva Centre for Emerging Viral, D. Novel SARS-CoV-2 variants: The pandemics within the pandemic. Clin. Microbiol. Infect. 2021, 27, 1109–1117. [Google Scholar] [CrossRef]
  8. Udugama, B.; Kadhiresan, P.; Kozlowski, H.N.; Malekjahani, A.; Osborne, M.; Li, V.Y.C.; Chen, H.; Mubareka, S.; Gubbay, J.B.; Chan, W.C.W. Diagnosing COVID-19: The Disease and Tools for Detection. ACS Nano 2020, 14, 3822–3835. [Google Scholar] [CrossRef] [Green Version]
  9. Lim, H.J.; Baek, Y.H.; Park, M.Y.; Yang, J.H.; Kim, M.J.; Sung, N.; Sohn, Y.H.; Lee, S.H.; Park, J.E.; Yang, Y.J. Performance Analysis of Self-Collected Nasal and Oral Swabs for Detection of SARS-CoV-2. Diagnostics 2022, 12, 2279. [Google Scholar] [CrossRef]
  10. Nunez, I.; Belaunzaran-Zamudio, P.F.; Caro-Vega, Y. Result Turnaround Time of RT-PCR for SARS-CoV-2 is the Main Cause of COVID-19 Diagnostic Delay: A Country-Wide Observational Study of Mexico and Colombia. Rev. Investig. Clin. 2022, 74, 71–80. [Google Scholar] [CrossRef]
  11. Gourgeon, A.; Soulier, A.; Audureau, É.; Khouider, S.; Galbin, A.; Langlois, C.; Bouvier-Alias, M.; Rodriguez, C.; Chevaliez, S.; Pawlotsky, J.M.; et al. Performance of 22 Rapid Lateral Flow Tests for SARS-CoV-2 Antigen Detection and Influence of "Variants of Concern": Implications for Clinical Use. Microbiol. Spectr. 2022, 10, e0115722. [Google Scholar] [CrossRef]
  12. Corman, V.M.; Haage, V.C.; Bleicker, T.; Schmidt, M.L.; Muhlemann, B.; Zuchowski, M.; Jo, W.K.; Tscheak, P.; Moncke-Buchner, E.; Muller, M.A.; et al. Comparison of seven commercial SARS-CoV-2 rapid point-of-care antigen tests: A single-centre laboratory evaluation study. Lancet Microbe 2021, 2, e311–e319. [Google Scholar] [CrossRef] [PubMed]
  13. Liu, Y.; Zhan, L.; Qin, Z.; Sackrison, J.; Bischof, J.C. Ultrasensitive and Highly Specific Lateral Flow Assays for Point-of-Care Diagnosis. ACS Nano 2021, 15, 3593–3611. [Google Scholar] [CrossRef] [PubMed]
  14. Seitz, T.; Lickefett, B.; Traugott, M.; Pawelka, E.; Karolyi, M.; Baumgartner, S.; Jansen-Skoupy, S.; Atamaniuk, J.; Fritsche-Polanz, R.; Asenbaum, J.; et al. Evaluation of Five Commercial SARS-CoV-2 Antigen Tests in a Clinical Setting. J. Gen. Intern. Med. 2022, 37, 1494–1500. [Google Scholar] [CrossRef]
  15. Scheiblauer, H.; Filomena, A.; Nitsche, A.; Puyskens, A.; Corman, V.M.; Drosten, C.; Zwirglmaier, K.; Lange, C.; Emmerich, P.; Muller, M.; et al. Comparative sensitivity evaluation for 122 CE-marked rapid diagnostic tests for SARS-CoV-2 antigen, Germany, September 2020 to April 2021. Eurosurveillance 2021, 26, 2100441. [Google Scholar] [CrossRef] [PubMed]
  16. Lanser, L.; Bellmann-Weiler, R.; Ottl, K.W.; Huber, L.; Griesmacher, A.; Theurl, I.; Weiss, G. Evaluating the clinical utility and sensitivity of SARS-CoV-2 antigen testing in relation to RT-PCR Ct values. Infection 2021, 49, 555–557. [Google Scholar] [CrossRef]
  17. Čretnik, T.; Retelj, M.; Janežič, S.; Mahnič, A.; Tesovnik, T.; Šket, R.; Bizjan, B.J.; Jeverica, S.; Ravnik, M.; Rak, M.; et al. Accuracy of Allplex SARS-CoV-2 assay amplification curve analysis for the detection of SARS-CoV-2 variant Alpha. Future Microbiol. 2022, 17, 1125–1131. [Google Scholar] [CrossRef]
  18. Pavletic, M.; Mazor, M.; Lerga, M.; Mileta, T.; Zeleznjak, J.; Ruzic, T.; Ravlic, S.; Palcevski, D.; Kirincic, J.; Meznaric, S.; et al. Fast, Reliable, and Simple Point-of-Care-like Adaptation of RT-qPCR for the Detection of SARS-CoV-2 for Use in Hospital Emergency Departments. Viruses 2021, 13, 2413. [Google Scholar] [CrossRef]
  19. Lim, H.J.; Jung, H.S.; Park, M.Y.; Baek, Y.H.; Kannappan, B.; Park, J.Y.; Yang, J.H.; Seol, J.H.; Lee, M.W.; Jung, S.K.; et al. Evaluation of Three Automated Extraction Systems for the Detection of SARS-CoV-2 from Clinical Respiratory Specimens. Life 2022, 12, 68. [Google Scholar] [CrossRef]
  20. Kim, H.W.; Park, M.; Lee, J.H. Clinical Evaluation of the Rapid STANDARD Q COVID-19 Ag Test for the Screening of Severe Acute Respiratory Syndrome Coronavirus 2. Ann. Lab. Med. 2022, 42, 100–104. [Google Scholar] [CrossRef]
  21. Mayfield, J.A.; Ortiz, J.; Ledden, D.J. The Impact of Universal Transport Media and Viral Transport Media Liquid Samples on a SARS-CoV-2 Rapid Antigen Test. Arch. Intern. Med. Res. 2022, 5, 481–487. [Google Scholar] [CrossRef]
  22. Lim, H.J.; Park, M.Y.; Jung, H.S.; Kwon, Y.; Kim, I.; Kim, D.K.; Yu, N.; Sung, N.; Lee, S.H.; Park, J.E.; et al. Development of an efficient Sanger sequencing-based assay for detecting SARS-CoV-2 spike mutations. PLoS ONE 2021, 16, e0260850. [Google Scholar] [CrossRef] [PubMed]
  23. Boeschoten, S.; Catal, C.; Tekinerdogan, B.; Lommen, A.; Blokland, M. The automation of the development of classification models and improvement of model quality using feature engineering techniques. Expert Syst. Appl. 2023, 213, 118912. [Google Scholar] [CrossRef]
  24. Das, D.; Saikia, H.; Bora, D.; Bhattacharjee, D.; Das, J. A survival analysis approach for identifying the risk factors in time to recovery of COVID-19 patients using Cox proportional hazard model. Decis. Anal. J. 2022, 5, 100137. [Google Scholar] [CrossRef]
  25. Habibzadeh, F.; Habibzadeh, P.; Yadollahie, M. On determining the most appropriate test cut-off value: The case of tests with continuous results. Biochem. Med. 2016, 26, 297–307. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Zhu, Z.; Lian, X.; Su, X.; Wu, W.; Marraro, G.A.; Zeng, Y. From SARS and MERS to COVID-19: A brief summary and comparison of severe acute respiratory infections caused by three highly pathogenic human coronaviruses. Respir. Res. 2020, 21, 224. [Google Scholar] [CrossRef]
  27. He, D.; Cowling, B.J.; Ali, S.T.; Stone, L. Rapid global spread of Variant of Concern of SARS-CoV-2. IJID Reg. 2022; in press. [Google Scholar] [CrossRef]
  28. Sharma, S.; Shrivastava, S.; Kausley, S.B.; Rai, B.; Pandit, A.B. Coronavirus: A comparative analysis of detection technologies in the wake of emerging variants. Infection 2022, 51, 1–19. [Google Scholar] [CrossRef]
  29. Schuit, E.; Venekamp, R.P.; Hooft, L.; Veldhuijzen, I.K.; van den Bijllaardt, W.; Pas, S.D.; Zwart, V.F.; Lodder, E.B.; Hellwich, M.; Koppelman, M.; et al. Diagnostic accuracy of covid-19 rapid antigen tests with unsupervised self-sampling in people with symptoms in the omicron period: Cross sectional study. BMJ 2022, 378, e071215. [Google Scholar] [CrossRef]
  30. Alqahtani, M.; Abdulrahman, A.; Mustafa, F.; Alawadhi, A.I.; Alalawi, B.; Mallah, S.I. Evaluation of Rapid Antigen Tests Using Nasal Samples to Diagnose SARS-CoV-2 in Symptomatic Patients. Front. Public Health 2021, 9, 728969. [Google Scholar] [CrossRef]
  31. Chu, V.T.; Schwartz, N.G.; Donnelly, M.A.P.; Chuey, M.R.; Soto, R.; Yousaf, A.R.; Schmitt-Matzen, E.N.; Sleweon, S.; Ruffin, J.; Thornburg, N.; et al. Comparison of Home Antigen Testing With RT-PCR and Viral Culture During the Course of SARS-CoV-2 Infection. JAMA Intern. Med. 2022, 182, 701–709. [Google Scholar] [CrossRef]
  32. Cocherie, T.; Bastide, M.; Sakhi, S.; Zafilaza, K.; Flandre, P.; Leducq, V.; Jary, A.; Burrel, S.; Louet, M.; Calvez, V.; et al. Decreased Sensitivity of Rapid Antigen Test Is Associated with a Lower Viral Load of Omicron than Delta SARS-CoV-2 Variant. Microbiol. Spectr. 2022, 10, e0192222. [Google Scholar] [CrossRef] [PubMed]
  33. Schuit, E.; Veldhuijzen, I.K.; Venekamp, R.P.; van den Bijllaardt, W.; Pas, S.D.; Lodder, E.B.; Molenkamp, R.; GeurtsvanKessel, C.H.; Velzing, J.; Huisman, R.C.; et al. Diagnostic accuracy of rapid antigen tests in asymptomatic and presymptomatic close contacts of individuals with confirmed SARS-CoV-2 infection: Cross sectional study. BMJ 2021, 374, n1676. [Google Scholar] [CrossRef] [PubMed]
  34. Hong, K.H.; Kim, G.J.; Roh, K.H.; Sung, H.; Lee, J.; Kim, S.Y.; Kim, T.S.; Park, J.S.; Huh, H.J.; Park, Y.; et al. Update of Guidelines for Laboratory Diagnosis of COVID-19 in Korea. Ann. Lab. Med. 2022, 42, 391–397. [Google Scholar] [CrossRef] [PubMed]
  35. Fernandez-Montero, A.; Argemi, J.; Rodriguez, J.A.; Arino, A.H.; Moreno-Galarraga, L. Validation of a rapid antigen test as a screening tool for SARS-CoV-2 infection in asymptomatic populations. Sensitivity, specificity and predictive values. EClinicalMedicine 2021, 37, 100954. [Google Scholar] [CrossRef]
  36. Dong, L.; Li, W.F.; Jiang, Y. Performance evaluation of antigen detection rapid diagnostic test (Ag-RDT) for COVID-19 diagnosis in a primary healthcare center during the Shanghai COVID-19 quarantine period. Virol. J. 2022, 19, 140. [Google Scholar] [CrossRef]
Figure 1. Performance of the four RAT kits in detecting SARS-CoV-2 (based on Ct values). ROC curves showing the diagnostic performance of the (A) PanbioTM, (B) StandardTM, (C) Humasis, and (D) Acrosis kits. The colored lines represent the diagnostic performance of each RAT kit.
Figure 1. Performance of the four RAT kits in detecting SARS-CoV-2 (based on Ct values). ROC curves showing the diagnostic performance of the (A) PanbioTM, (B) StandardTM, (C) Humasis, and (D) Acrosis kits. The colored lines represent the diagnostic performance of each RAT kit.
Biomedicines 11 00701 g001
Table 1. Comparison of the diagnosis performance of the RAT kits.
Table 1. Comparison of the diagnosis performance of the RAT kits.
RATrRT-PCR
TP (n)FP (n)TN (n)FN (n)Precision (%)Recall (%)
PanbioTM441369836199.3254.99
StandardTM447170035599.7855.74
Humasis432070137010053.87
Acrosis614070118810076.56
Data indicate the number of samples or percentages. TP, true positive; FP, false positive; TN, true negative; FN, false negative.
Table 2. RAT results stratified using a detailed classification based on the Ct value.
Table 2. RAT results stratified using a detailed classification based on the Ct value.
RATrRT-PCR
TotalNeg.Ct < 2020 ≤ Ct < 2222 ≤ Ct < 2424 ≤ Ct < 2626 ≤ Ct < 2828 ≤ Ct < 30Ct ≥ 30
PanbioTM29.5%
(444/1503)
0.4%
(3/701)
100%
(137/137)
98.2%
(108/110)
89.6%
(86/96)
60.0%
(60/100)
29.4%
(30/102)
14.3%
(15/105)
3.3%
(5/152)
StandardTM29.8%
(448/1503)
0.1%
(1/701)
100%
(137/137)
100%
(110/110)
92.7%
(89/96)
61.0%
(61/100)
28.4%
(29/102)
12.4%
(13/105)
5.3%
(8/152)
Humasis28.7%
(432/1503)
0%
(0/701)
100%
(137/137)
98.2%
(108/110)
89.6%
(86/96)
59.0%
(59/100)
23.5%
(24/102)
12.4%
(13/105)
3.3%
(5/152)
Acrosis40.9%
(614/1503)
0%
(0/701)
100% (137/137)100%
(110/110)
100%
(96/96)
97.0%
(97/100)
95.1%
(97/102)
55.2%
(58/105)
12.5%
(19/152)
Data are presented as percentage (number of RAT-positive samples/rRT-PCR-positive samples). Neg., negative.
Table 3. Comparison of the true-positive results for RAT kits.
Table 3. Comparison of the true-positive results for RAT kits.
Comparison KitsNDifferencep
PanbioTM vs. StandardTM1503–40.82
PanbioTM vs. Humasis1503120.74
PanbioTM vs. Acrosis1503–170<0.001 *
StandardTM vs. Humasis1503160.58
StandardTM vs. Acrosis1503–166<0.001 *
Humasis vs. Acrosis1503–182<0.001 *
The p-value represents the difference between the two curves according to the log-rank test results. * p < 0.05.
Table 4. True-positive values of the RAT kits according to the SARS-CoV-2 lineages.
Table 4. True-positive values of the RAT kits according to the SARS-CoV-2 lineages.
VariantsLineageNRapid Antigen Test
PanbioTMStandardTMHumasisAcrosis
Pre-DeltaB.1.49788/88/88/88/8
B.1.61933/33/33/33/3
B.1.62042/4 *3/4 *2/4 *4/4
DeltaAY.6933/33/33/33/3
AY.122.544/44/44/44/4
OmicronBA.122/22/22/22/2
BA.21717/1717/1717/1717/17
BA.411/11/11/11/1
BA.53331/33 *31/33 *30/33 *33/33
BE.122/22/22/22/2
Data indicate the number of RAT-positive/rRT-PCR-positive samples. * False-negative result was observed in SARS-CoV-2-positive samples.
Table 5. Comparison of N mutations in the B.1.620 and BA.5 lineages.
Table 5. Comparison of N mutations in the B.1.620 and BA.5 lineages.
LineageNResultCt ValueMutation
CommonAdditional
B.1.6201Con.17.07A220F-
1Con.22.55L221F
2Dis.21.34, 22.19L221F
BA.528Con.16.06–24.88ERS31–33del,
P13L, R203K,
G204R, S413R
-
3Dis.18.63–24.40-
2Con.22.99, 23.37A182T
Data indicate the number of concordant and discordant samples with the corresponding mutations. Con., concordant; Dis., discordant.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lim, H.-J.; Park, M.-Y.; Baek, Y.-H.; Lee, H.-S.; Kim, I.; Kwon, Y.; You, Y.; Nam, K.; Yang, J.-H.; Kim, M.-J.; et al. Evaluation of Four Rapid Antigen Tests for the Detection of SARS-CoV-2 Infection with Nasopharyngeal Swabs. Biomedicines 2023, 11, 701. https://doi.org/10.3390/biomedicines11030701

AMA Style

Lim H-J, Park M-Y, Baek Y-H, Lee H-S, Kim I, Kwon Y, You Y, Nam K, Yang J-H, Kim M-J, et al. Evaluation of Four Rapid Antigen Tests for the Detection of SARS-CoV-2 Infection with Nasopharyngeal Swabs. Biomedicines. 2023; 11(3):701. https://doi.org/10.3390/biomedicines11030701

Chicago/Turabian Style

Lim, Ho-Jae, Min-Young Park, Young-Hyun Baek, Hyeon-Seo Lee, Inhee Kim, Youngjin Kwon, Youngshin You, Kyoungwoo Nam, Jae-Hyun Yang, Min-Jin Kim, and et al. 2023. "Evaluation of Four Rapid Antigen Tests for the Detection of SARS-CoV-2 Infection with Nasopharyngeal Swabs" Biomedicines 11, no. 3: 701. https://doi.org/10.3390/biomedicines11030701

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop