Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

An international, interlaboratory ring trial confirms the feasibility of an extraction-less “direct” RT-qPCR method for reliable detection of SARS-CoV-2 RNA in clinical samples

  • Margaret G. Mills ,

    Contributed equally to this work with: Margaret G. Mills, Emily Bruce

    Roles Formal analysis, Investigation, Methodology

    Affiliation Virology Division, Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, United States of America

  • Emily Bruce ,

    Contributed equally to this work with: Margaret G. Mills, Emily Bruce

    Roles Conceptualization, Formal analysis, Investigation, Writing – original draft, Writing – review & editing

    Affiliation Division of Immunobiology, Department of Medicine, Robert Larner, M.D. College of Medicine, University of Vermont, Burlington, Vermont, United States of America

  • Meei-Li Huang,

    Roles Investigation, Methodology

    Affiliation Virology Division, Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, United States of America

  • Jessica W. Crothers,

    Roles Investigation

    Affiliation Division of Immunobiology, Department of Medicine, Robert Larner, M.D. College of Medicine, University of Vermont, Burlington, Vermont, United States of America

  • Ollivier Hyrien,

    Roles Data curation, Formal analysis, Methodology

    Affiliation Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, Washington, United States of America

  • Christopher A. L. Oura,

    Roles Investigation, Supervision

    Affiliation School of Veterinary Medicine, University of the West Indies, St. Augustine, Trinidad and Tobago

  • Lemar Blake,

    Roles Investigation

    Affiliation School of Veterinary Medicine, University of the West Indies, St. Augustine, Trinidad and Tobago

  • Arianne Brown Jordan,

    Roles Investigation

    Affiliation Ministry of Health, Port of Spain, Trinidad and Tobago

  • Susan Hester,

    Roles Conceptualization, Investigation, Methodology, Writing – review & editing

    Affiliation Office of Research and Development, U.S. Environmental Protection Agency, Research Triangle Park, North Carolina, United States of America

  • Leah Wehmas,

    Roles Conceptualization, Investigation, Methodology, Writing – review & editing

    Affiliation Office of Research and Development, U.S. Environmental Protection Agency, Research Triangle Park, North Carolina, United States of America

  • Bernard Mari,

    Roles Formal analysis, Investigation, Methodology, Writing – review & editing

    Affiliation Université Côte d’Azur, CNRS, Institut de Pharmacologie Moléculaire et Cellulaire, FHU-OncoAge, Valbonne, France

  • Pascal Barby,

    Roles Conceptualization, Investigation, Methodology, Writing – review & editing

    Affiliation Université Côte d’Azur, CNRS, Institut de Pharmacologie Moléculaire et Cellulaire, FHU-OncoAge, Valbonne, France

  • Caroline Lacoux,

    Roles Investigation

    Affiliation Université Côte d’Azur, CNRS, Institut de Pharmacologie Moléculaire et Cellulaire, FHU-OncoAge, Valbonne, France

  • Julien Fassy,

    Roles Investigation

    Affiliation Université Côte d’Azur, CNRS, Institut de Pharmacologie Moléculaire et Cellulaire, FHU-OncoAge, Valbonne, France

  • Pablo Vial,

    Roles Investigation, Methodology

    Affiliation Facultad de Medicina Clínica Alemana Universidad del Desarrollo, Programa Hantavirus, Instituto de Ciencias e Innovación en Medicina, Santiago, Chile

  • Cecilia Vial,

    Roles Investigation, Methodology

    Affiliation Facultad de Medicina Clínica Alemana Universidad del Desarrollo, Programa Hantavirus, Instituto de Ciencias e Innovación en Medicina, Santiago, Chile

  • Jose R. W. Martinez,

    Roles Investigation

    Affiliation Facultad de Medicina Clínica Alemana Universidad del Desarrollo, Programa Hantavirus, Instituto de Ciencias e Innovación en Medicina, Santiago, Chile

  • Olusola Olalekan Oladipo,

    Roles Data curation, Investigation, Methodology

    Affiliation Biochemistry Division, National Veterinary Research Institute, Vom, Nigeria

  • Bitrus Inuwa,

    Roles Investigation, Methodology

    Affiliation Infectious and Transboundary Animal Diseases, National Veterinary Research Institute, Vom, Nigeria

  • Ismaila Shittu,

    Roles Investigation, Methodology

    Affiliation Infectious and Transboundary Animal Diseases, National Veterinary Research Institute, Vom, Nigeria

  • Clement A. Meseko,

    Roles Investigation, Methodology

    Affiliation Infectious and Transboundary Animal Diseases, National Veterinary Research Institute, Vom, Nigeria

  • Roger Chammas,

    Roles Data curation, Investigation, Methodology, Supervision, Writing – review & editing

    Affiliation Faculdade de Medicina da Universidade de São Paulo, Departamento de Radiologia e Oncologia, Centro de Investigação Translacional em Oncologia, Instituto do Cancer do Estado de São Paulo, São Paulo, Brazil

  • Carlos Ferreira Santos,

    Roles Investigation, Methodology

    Affiliation Department of Biological Sciences, Bauru School of Dentistry, University of São Paulo, Bauru, São Paulo, Brazil

  • Thiago José Dionísio,

    Roles Investigation

    Affiliation Department of Biological Sciences, Bauru School of Dentistry, University of São Paulo, Bauru, São Paulo, Brazil

  • Thais Francini Garbieri,

    Roles Investigation

    Affiliation Department of Biological Sciences, Bauru School of Dentistry, University of São Paulo, Bauru, São Paulo, Brazil

  • Viviane Aparecida Parisi,

    Roles Investigation

    Affiliation Department of Biological Sciences, Bauru School of Dentistry, University of São Paulo, Bauru, São Paulo, Brazil

  • Maria Cassia Mendes-Correa,

    Roles Investigation

    Affiliation Department of Infectious Diseases, Institute of Tropical Medicine of São Paulo, São Paulo, Brazil

  • Anderson V. de Paula,

    Roles Investigation

    Affiliation Department of Infectious Diseases, Institute of Tropical Medicine of São Paulo, São Paulo, Brazil

  • Camila M. Romano,

    Roles Data curation, Investigation

    Affiliation Department of Infectious Diseases, Institute of Tropical Medicine of São Paulo, São Paulo, Brazil

  • Luiz Gustavo Bentim Góes,

    Roles Data curation, Investigation, Methodology

    Affiliations Scientific Platform Pasteur-USP, Universidade de São Paulo, São Paulo, Brazil, Department of Microbiology, Biomedical Sciences Institute, University of São Paulo, São Paulo, Brazil

  • Paola Minoprio,

    Roles Investigation

    Affiliation Scientific Platform Pasteur-USP, Universidade de São Paulo, São Paulo, Brazil

  • Angelica C. Campos,

    Roles Investigation

    Affiliation Scientific Platform Pasteur-USP, Universidade de São Paulo, São Paulo, Brazil

  • Marielton P. Cunha,

    Roles Investigation

    Affiliation Scientific Platform Pasteur-USP, Universidade de São Paulo, São Paulo, Brazil

  • Ana Paula P. Vilela,

    Roles Investigation

    Affiliation Scientific Platform Pasteur-USP, Universidade de São Paulo, São Paulo, Brazil

  • Tonney Nyirenda,

    Roles Data curation, Investigation, Methodology

    Affiliation Department of Pathology, College of Medicine, University of Malawi, Blantyre, Malawi

  • Rajhab Sawasawa Mkakosya,

    Roles Data curation, Investigation, Methodology

    Affiliation Department of Pathology, College of Medicine, University of Malawi, Blantyre, Malawi

  • Adamson S. Muula,

    Roles Data curation, Investigation, Methodology, Project administration

    Affiliation Department of Public Health, College of Medicine, University of Malawi, Blantyre, Malawi

  • Rebekah E. Dumm,

    Roles Data curation, Formal analysis, Investigation, Methodology, Validation

    Affiliation Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, United States of America

  • Rebecca M. Harris,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology

    Affiliation Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, United States of America

  • Constance A. Mitchell,

    Roles Data curation, Project administration, Supervision, Writing – original draft, Writing – review & editing

    Affiliation Health and Environmental Sciences Institute, Washington, DC, United States of America

  • Syril Pettit ,

    Roles Conceptualization, Data curation, Project administration, Supervision, Writing – original draft, Writing – review & editing

    spettit@hesiglobal.org (SP); jason.botten@uvm.med.edu (JB); kjerome@uw.edu (KRJ)

    Affiliation Health and Environmental Sciences Institute, Washington, DC, United States of America

  • Jason Botten ,

    Roles Conceptualization, Data curation, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

    spettit@hesiglobal.org (SP); jason.botten@uvm.med.edu (JB); kjerome@uw.edu (KRJ)

    Affiliations Division of Immunobiology, Department of Medicine, Robert Larner, M.D. College of Medicine, University of Vermont, Burlington, Vermont, United States of America, Department of Microbiology and Molecular Genetics, Robert Larner, M.D. College of Medicine, University of Vermont, Burlington, Vermont, United States of America

  •  [ ... ],
  • Keith R. Jerome

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing

    spettit@hesiglobal.org (SP); jason.botten@uvm.med.edu (JB); kjerome@uw.edu (KRJ)

    Affiliations Virology Division, Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, United States of America, Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, Washington, United States of America

  • [ view all ]
  • [ view less ]

Abstract

Reverse transcription–quantitative polymerase chain reaction (RT-qPCR) is used worldwide to test and trace the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). “Extraction-less” or “direct” real time–reverse transcription polymerase chain reaction (RT-PCR) is a transparent and accessible qualitative method for SARS-CoV-2 detection from nasopharyngeal or oral pharyngeal samples with the potential to generate actionable data more quickly, at a lower cost, and with fewer experimental resources than full RT-qPCR. This study engaged 10 global testing sites, including laboratories currently experiencing testing limitations due to reagent or equipment shortages, in an international interlaboratory ring trial. Participating laboratories were provided a common protocol, common reagents, aliquots of identical pooled clinical samples, and purified nucleic acids and used their existing in-house equipment. We observed 100% concordance across laboratories in the correct identification of all positive and negative samples, with highly similar cycle threshold values. The test also performed well when applied to locally collected patient nasopharyngeal samples, provided the viral transport media did not contain charcoal or guanidine, both of which appeared to potently inhibit the RT-PCR reaction. Our results suggest that direct RT-PCR assay methods can be clearly translated across sites utilizing readily available equipment and expertise and are thus a feasible option for more efficient COVID-19 coronavirus disease testing as demanded by the continuing pandemic.

1. Introduction

The global coronavirus disease (COVID-19) pandemic response depends on effective rollout of recently approved vaccines and the use of nonpharmaceutical interventions to slow the spread of the disease. Physical distancing supported by test-and-trace informed containment strategies has been promoted worldwide [1]. The effectiveness of testing as a containment strategy requires the implementation of accessible, affordable, reliable, and rapidly executable test methods that can meet the rapid pace of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) transmission [24]. At present, this goal remains largely unmet.

The majority of regional and national health laboratories around the world rely on reverse transcription–quantitative polymerase chain reaction (RT-qPCR) SARS-CoV-2 virologic testing methods such as those developed by the World Health Organization and the U.S. Centers for Disease Control and Prevention (CDC) to support their public health programs [5,6]. The methods themselves are robust and have proven to be useful standards for detection and reporting. However, sample processing time and a lack of supplies to support extraction as required to run this type of assay have resulted in widely reported backlogs and shortages in the United States and around the world [7]. In regions that also suffer from systemic financial and logistical challenges (e.g., Africa, the Caribbean, and South America), these hurdles will continue to consistently impair reliable procurement of consumables, support for staffing, and thus testing viability [8,9]. Although the diversity and efficiency of commercial virologic and serologic test methods expands weekly, most public health laboratories lack the resources (human and capital) or remit to pivot to novel commercial methods.

To address these challenges, the nonprofit Health and Environmental Science Institute (HESI) convened an international network of public and academic COVID-19 testing laboratories—the Propagate Network—with the goal of collectively evaluating and disseminating practical, efficient, and impactful transparent and accessible methods for SARS-CoV-2 detection. The Propagate Network and others have identified “extraction-less” or “direct” real time–reverse transcription polymerase chain reaction (RT-PCR) as a transparent and accessible qualitative method for SARS-CoV-2 detection from nasopharyngeal samples with the potential to generate actionable data more quickly, at a lower cost, and with fewer experimental resources than full RT-qPCR [10,11]. The method allows for detection of SARS-CoV-2 viral ribonucleic acid (RNA) with the omission of the most labor-intensive step—the RNA extraction step—and its associated extraction reagents. Published intralaboratory studies indicate that the technique is internally reproducible (with some loss of sensitivity compared to standard RT-PCR) and is effective in detecting both true negatives and positives. Notably, direct RT-qPCR remains sufficiently sensitive to detect viral RNA from patients most likely to be infectious (cycle threshold [Ct] < 33) [1215].

The major goal of this Propagate Network study was to determine the practical utility of a transparent and accessible, direct RT-PCR assay [10] via an international, interlaboratory ring trial. The study engaged 10 global sites, including laboratories currently experiencing many of the testing limitations described above, in a series of studies involving a common protocol, common reagents, aliquots of identical pooled clinical samples, and purified nucleic acids, using their existing in-house equipment. Our results suggest that this transparent and accessible, direct RT-PCR assays are a feasible option for more efficient COVID-19 testing as demanded by the growing pandemic.

2. Materials and methods

2.1 Participants

The Propagate Network study was coordinated and partially funded by the international nonprofit HESI as part of its global public health mission and via voluntary contributions of time and effort from the participating partners. Special acknowledgment is given to the University of Washington Virology Laboratory (UWVL) for their efforts to prepare and ship samples for this study and to the University of Vermont Larner School of Medicine for their support in refining the study protocols and recruiting partner laboratories.

Ten laboratories were recruited to participate in the trial for the detection of SARS-CoV-2 RNA from patient nasopharyngeal swabs without RNA extraction using kits provided by UWVL (Table 1). Laboratories participated voluntarily and were not offered any compensation for their participation. Due to logistical shipping challenges, which were in large part brought on by the pandemic, samples were unable to be sent to Malawi or Nigeria, underlying the hardships some areas face when testing relies on reagents or materials from other countries.

thumbnail
Table 1. Institutions and countries of the laboratories participating in the trial.

https://doi.org/10.1371/journal.pone.0261853.t001

2.2 Ethical statement

Use of the samples was determined to be exempt under UW institutional guidelines because they were de-identified and pooled prior to inclusion in the test kits, and therefore were not considered human subjects because they contained no individually identifiable material. For Project C, participating laboratories sought the locally appropriate review and permissions for use of de-identified clinical samples as described below.

  • University of West Indies: Based on the Campus Research Ethics Committee at the University of West Indies, this research met the criteria for Exemption. This decision was made by the chair of the ethics committee, Professor Jerome De Liste.
  • Department of Infectious Diseases, Institute of Tropical Medicine of São Paulo: Sample use was approved by the local ethics committee (Comissão de Ética para Análise de Projeto de Pesquisa; protocol number CAAE 30419320.7.0000.0068). Informed consent was obtained from all the individuals enrolled in this study.
  • National Veterinary Research Institute, Nigeria: This work and samples were approved for ethical use within the emergency response to COVID-19 control in Nigeria through rapid laboratory diagnosis. The National Veterinary Research Institute in accordance with the World Organization for Animal Health guidance offered its facility for Public Health Service in Nigeria following activation by the Nigerian Centre for Diseases Control of the Federal Ministry of Health.
  • University of Malawi: Based on a review by the College of Medicine Research and Ethics Committee (COMREC), no IRB approval acknowledgments were required. This was reviewed by the COMREC administrator, Dr. Lucinda Manda-Taylor, and the compliance officer, Khama Mita.
  • University of Vermont: This work was approved under a waiver of consent by the University of Vermont IRB (STUDY00000881).
  • Instituto de Ciencias e Innovación en Medicina, Facultad de Medicina Clínica Alemana Universidad del Desarrollo, Santiago, Chile: The study and publication of its data was approved by a decision made by the Scientific Ethics Committee at the Center of Bioethics at the College of Medicine on 17 March 2021 (as approved by Dr. Marcial Osorio (President of the Scientific Ethics Committee) and Javiera Bellolio A. (Executive Secretary of the Scientific Ethics Committee, College of Medicine, Center of Bioethics) upon review of the document “Admin_Comp_200–53 Requests for Release of Clinical Specimens or Results”, as well as this manuscript. The committee made the statement “It is considered that, given the modality of the study, where the identity is duly protected, and given the importance from the public health point of view, this project has been approved by the Committee for the publication of the data.” The signed document can be provided upon request.
  • Bauru School of Dentistry, Department of Biological Sciences, University of Sao Paulo: Sample use approved by the Institutional Review Board of the Bauru School of Dentistry, University of Sao Paulo (CAAE # 32658720.4.0000.5417)

2.3 Study design

The Propagate ring trial consisted of three components. Projects A and B engaged participant laboratories in the analysis of pooled samples disseminated from the lead laboratory (UWVL) for the purpose of evaluating the cross-laboratory performance of the direct method with parallel (identical) samples. Project C characterized the feasibility of the direct method as applied to locally sourced samples collected as part of regional public health testing efforts (Fig 1).

All laboratories were invited to participate in Projects A, B, and C. Logistical challenges due to COVID-related shipping restrictions prohibited the involvement of the Malawi and Nigeria laboratories in Projects A and B.

Project A.

To confirm that all reagents arrived safely and that every laboratory could perform the direct RT-PCR method, each laboratory tested a set of eight nucleic acid samples purified from patients with COVID-19 and supplied by UWVL, which included six blinded samples (three positive and three negative), one sample identified positive, and one sample identified negative to serve as controls, plus a laboratory-supplied no-template water control. Each laboratory reviewed the results of Project A with the study coordinator to confirm that they had correctly identified 100% of the positive and negative blinded samples before proceeding.

Project B.

Each laboratory then tested a set of 34 samples supplied by UWVL, including 30 blinded samples (25 positive CT and 5 negative) as well as 2 identified positive and 2 identified negative samples as controls, plus a laboratory-supplied no-template water control. Results for all Project B samples were shared with the study coordinator.

Project C.

When and where possible, laboratories selected known-positive, locally collected clinical samples and known-negative samples and tested each by both their standard extraction method and the direct RT-PCR method. Results for all Project C samples were shared with the study coordinator.

2.4 Heat inactivation

To validate whether a 10-min heating step would inactivate infectious SARS-CoV-2 virus present in clinical samples, the University of Vermont team incubated high-titer stocks of authentic SARS-CoV-2 at 95°C (in a heat block) or room temperature in 1.5-ml Eppendorf tubes for 10 min, spun samples briefly in a microcentrifuge, then measured the infectious units remaining in the heat-treated samples versus the untreated controls by immunofocus assay [16]. The stock virus (strain 2019‐nCoV/USA_USA‐WA1/2020 [WA1]) was graciously provided by Kenneth Plante and the World Reference Center for Emerging Viruses and Arboviruses at the University of Texas Medical Branch.

2.5 Sample preparation

Identical kits were prepared by UWVL and sent on dry ice to each participating laboratory. These kits included reagents for the three possible projects (Fig 1): blinded purified total nucleic acid samples plus positive and negative controls (Project A); blinded unpurified patient samples plus positive and negative controls (Project B); and sufficient enzyme, buffer, primers, and probes to test all samples for Projects A and B, as well as local patient samples (Project C).

To make positive and negative control samples for the kits, nasopharyngeal patient samples in viral transport media were gathered from UWVL’s clinical specimen collection. Three positive samples with a high concentration of SARS-CoV-2 viral RNA (Ct ~15) were pooled together and then diluted 1:32 in a pool of negative samples. Aliquots of positive pooled samples (Ct ~20) and of negative pooled samples were subjected to RNA extraction using a MagNA Pure LC (Roche) to generate purified positive and negative total nucleic acid for Project A. A total of four 50-μl aliquots each of positive and negative were included in each kit for Project A: one identified positive and negative control, and six blinded samples.

The unpurified positive pool was diluted further in the negative pool to generate samples at 18 specific expected Ct values ranging from 21 to 32 for Project B. One 50-μl aliquot each of 11 of these samples, and two 50-μl aliquots each of seven of the samples (expected Ct 28–31) were included in the kit, for a total of 25 positive samples. Two identified negative and two identified positive aliquots (Ct ~ 21) were included as controls in each kit for Project B. A random number generator was used to determine the order of blinded samples within the kits. All samples were tested in parallel by direct RT-PCR method and by MagNA Pure LC nucleic acid extraction followed by RT-PCR at UWVL to confirm negative samples and Ct range of positive samples.

2.6 Testing method

The direct RT-PCR method is described in Bruce et al. [10]. Briefly, 20 μl of each sample was heat treated for 10 min at 95°C then vortexed and spun down. A Master Mix was made by combining 7 μl of water, 12.5 μl of buffer mix, 1.5 μl of primer/probe mix (IDT), and 1 μl of AgPath-ID enzyme (ThermoFisher) per reaction. Either in 96-well optical PCR plates or optical strip tubes, 22 μl of Master Mix and 3 μl of heat-treated sample was added to each well or tube. All manipulations of clinical samples (transfer for heat inactivation as well as loading of the RT-PCR plate) were performed in a class IIA biosafety cabinet following biosafety level 2 practices. The plates or tubes were then covered with an optical adhesive cover or caps and spun down at 1000 rpm for 1 min. The RT-PCR reaction consisted of 10 min at 48°C for reverse transcription, 10 min at 95°C, and 40 cycles of 95°C for 15 s, followed by 60°C for 45 s with fluorescence measured at the end of each cycle. All samples were tested in duplicate, with water controls on each plate. Reactions to measure the SARS-CoV-2 N-gene (using CDC N2 primers and FAM-labeled probe) and human RNase P gene (using CDC RP primers and FAM-labeled probe) were carried out for each sample in parallel.

2.7 Data collection and analysis

For each sample, a mean Ct value was computed by averaging individual Ct values from all laboratories. A Ct value residual (for a given laboratory and sample) was defined as the individual Ct value minus the associated mean Ct value. For data visualization, individual Ct values and residual Ct values were plotted against mean Ct values. Assay specificity and sensitivity was evaluated using the negative and positive blinded samples.

3. Results

3.1 Heat inactivation

High-titer stocks of SARS-CoV-2 were treated at 95°C for 10 min. The stock virus had a titer of >106 focus forming units (FFUs) per milliliter. After heat treatment there was more than a 5-log drop, with no detectable foci after 10 min at 95°C (Fig 2).

thumbnail
Fig 2. Heat treatment inactivates SARS-CoV-2.

Stocks of SARS-CoV-2 split into aliquots were treated at 95°C for 10 min or untreated. Viral titer was determined using a focus forming assay with an antibody recognizing the viral N protein. Measurements are in focus forming units (FFU) per milliliter (n = 4 replicates). The limit of detection for this assay was 20 FFU/ml.

https://doi.org/10.1371/journal.pone.0261853.g002

3.2 Project A: Laboratory qualification

All participating laboratories correctly identified 100% of the positive (n = 3) and negative (n = 3) blinded samples sent for purposes of confirming that the samples arrived safely and that the laboratory was able to run the direct RT-PCR method.

3.3 Project B: Interlaboratory agreement using blinded samples

Qualitative agreement between laboratories.

The most critical performance measures of a SARS-CoV-2 test are its sensitivity and specificity (simply put, the ability to accurately distinguish the presence versus absence of the viral RNA) and the consistency of its performance across laboratories. As an initial approach, assay specificity and sensitivity was evaluated using 5 known-negative and 25 known-positive samples that were tested in a blinded fashion by the 10 laboratories. For the five negative samples, a total of 50 values were reported by the laboratories, all of which were reported as negative for virus. Thus, the assay demonstrated consistently high (100%) specificity across the laboratories for negative samples (Table 2). For the 25 positive samples, the 10 laboratories reported a total of 250 Ct values. All of these but one were reported as positive. Thus, all 10 laboratories were able to correctly detect the virus in 24 of 25 samples, and 9 of 10 laboratories were able to correctly detect the virus in all 25 samples using the assay, yielding consistently high [99.6% = (249/250) × 100%] sensitivity across the 10 laboratories for positive samples.

thumbnail
Table 2. Specificity and sensitivity of the test based on results from 30 blinded samples (5 negative and 25 positive) and the 10 laboratories.

https://doi.org/10.1371/journal.pone.0261853.t002

Quantitative agreement between laboratories.

In addition to providing a qualitative determination of the presence versus absence of virus, RT-PCR tests for SARS-CoV-2 can provide additional value by reporting their Ct value, which serves as a proxy for the amount of viral RNA present. We therefore investigated the Ct values reported for the blinded positive samples tested by the participating laboratories. In general, the agreement between Ct values from different laboratories was good, with tighter agreement at lower average Ct (higher viral loads) than at higher average Ct (lower viral loads; Fig 3).

thumbnail
Fig 3. High interlaboratory agreement in Ct values for blinded clinical samples in Project B.

Ct values plotted against the sample-specific average Ct values from the 10 laboratories. The solid line indicates the line of equation y = x. Each color indicates one laboratory; the same color is used to identify each laboratory in all subsequent figures.

https://doi.org/10.1371/journal.pone.0261853.g003

We then evaluated the overall quantitative performance of each individual laboratory against the sample-specific average Ct value as determined by all 10 laboratories. The Ct value residual for a given laboratory and sample was defined as the Ct value for the corresponding laboratory and sample minus the sample-specific average Ct value; the narrower their distribution within a laboratory, the more consistent the relationship of Ct values from the laboratory with the average Ct value from all laboratories. Residual Ct values had overall similar variability across samples and were minimally affected by the actual viral load (Fig 4). The residuals appeared to be centered around zero for most laboratories (Fig 5), with the exception of laboratories 3 and 7, for which residuals appeared systematically negative (indicative of Ct values consistently lower than average), and laboratory 9, in which residuals tended to be positive (indicative of Ct values consistently higher than average). There was no evidence that the assay had a lower sensitivity in this particular laboratory.

thumbnail
Fig 4. Distribution of Ct residuals relative to sample-specific average Ct values.

Residuals were consistent between laboratories regardless of viral load. Residuals were plotted against sample-specific average Ct values. Each colored dot represents a different participating laboratory per legend.

https://doi.org/10.1371/journal.pone.0261853.g004

thumbnail
Fig 5. Distribution of Ct value residuals plotted by laboratory.

Ct value residuals were calculated as the Ct values minus the sample-specific average Ct value.

https://doi.org/10.1371/journal.pone.0261853.g005

3.4 Project C: Application of direct RT-PCR on locally collected clinical samples

Seven Propagate partner laboratories conducted side-by-side comparisons of direct RT-PCR and extraction RT-PCR on clinical samples collected from their regions. For four laboratories (2, 7, 8, and 10), these studies demonstrated average losses of sensitivity of between 1.5 and 3.8 cycles in RP Ct value (Fig 6A; Table 3) and between 2.6 and 4.8 cycles in N2 Ct value (Fig 6B; Table 3), compared with direct RT-PCR. For RP, this resulted in no failure to detect any sample from any of the four laboratories. For samples in which N2 was detectable by both direct and extraction RT-PCR, the difference in Ct values between the two methods did not correlate with the Ct value obtained by either method (Fig 6C). However, a few samples (6 of 93) that were detected between Ct values of 28 and 39 by extraction RT-PCR were undetectable by direct RT-PCR, while other samples in that range were still detectable by the same laboratories (Fig 6C).

thumbnail
Fig 6. Comparison of direct versus extracted PCR analyses for locally sourced clinical samples (Project C).

The left (A) and center (B) boxplots illustrate the difference between direct and extracted Ct values in the five laboratories participating in Project C for the RP and N2 genes respectively. The right boxplot (C) illustrates the difference between direct and extracted Ct values plotted against the extracted Ct value, in five laboratories.

https://doi.org/10.1371/journal.pone.0261853.g006

thumbnail
Table 3. Difference in sensitivity of direct RT-PCR and extraction RT-PCR methods on local samples from five laboratories.

https://doi.org/10.1371/journal.pone.0261853.t003

For laboratory 9, direct RT-PCR yielded lower Ct values for both RP and N2 than extraction RT-PCR, with an average difference of −0.3 cycles and −1.6 cycles, respectively (Table 3). The reason for this unexpected result is not clear, but it may have been related to the laboratory’s observation that samples became “highly viscous” after the heating step (an observation not reported in any of the other participating laboratories). As with any method, the authors recommend internal validation of the approach prior to clinical implementation.

For two laboratories, direct amplification of both N2 and RP was unsuccessful in all samples, including for samples with low N2 Ct values (high viral loads) as measured by extraction RT-PCR. These samples were later determined to have been collected in transport media containing ingredients that were inhibitory to PCR, including charcoal and guanidine (e.g., ManTacc UTM and Jiangsu Rongye Technology LinkGen media were reported as incompatible). Information on the brand/type of viral transport media was not available for all samples used in Project C (this information is often not reported with swabbed samples as provided to analysis laboratories). However, the following media were specifically identified as compatible with this method (Hardy viral transport media, saline, and phosphate-buffered saline).

Overall, the direct approach worked effectively to detect samples deemed positive by standard RT-qPCR when samples were collected in media lacking charcoal or guanine.

4. Discussion

This study confirms that the direct RT-qPCR method, initially described by Bruce et al. [10], has the potential to meaningfully contribute to global efforts to detect and contain the COVID-19 pandemic. This study provides evidence that the direct RT-qPCR method is an efficient, reliable, and achievable method for detection of SARS-CoV-2. Although the reproducibility of the method has been reported in single-laboratory studies previously, this study is the first to demonstrate that a globally diverse set of laboratories operating with different equipment, clinical sample collection and handling conditions, resource limitations, and operating practices can successfully implement the method.

As described above, when centrally disseminated pooled samples were evaluated with a common Master Mix and primers/probes (Projects A and B), the Propagate partner laboratory results were >99.5% concordant (all negatives and all but one positive correctly identified and strong agreement on Ct values). This result demonstrates the robustness of the methodology. Although all Propagate partner laboratories had prior experience with standard RNA extraction RT-PCR analysis of SARS-CoV-2 test samples, they were able to adopt and implement the direct method on Project A and B samples with only a minimum of instruction (a brief written protocol and a few minutes of discussions via web meeting), showing that the method is easily transferable. Due to halts and delays in air shipments, the partner laboratories in Malawi and Nigeria were unable to receive or analyze the Project A/B sample kits. While this was unfortunate, it is emblematic of the challenges that the African continent (among others) continues to face in receiving needed laboratory supplies and the importance of resource-sensitive methods development efforts such as these.

In Project C of this study, Propagate partner laboratories were encouraged to use their own extraction methods and locally collected samples with RT-qPCR reagents supplied by UWVL to compare results from the direct method versus standard extraction-based PCR. The majority of the participating partner laboratories were successfully able to apply the method and reliably detect RT-PCR-positive samples. The importance of “ground testing” new methods was made evident when some of the laboratories were unable to detect any signal (N2 or RP) following the direct method despite using high-titer positive samples as detected by standard methods. Laboratories experiencing this problem in some cases were working with samples collected in commercial viral transport media that were often found to contain charcoal or in inactivating media such as those containing guanidine. We hypothesize that these are inhibitory to the RT-PCR reaction in the absence of an extraction phase. Similar inhibitory outcomes have been subsequently identified by other laboratories [11,17]. In other cases, the constituents of media were unknown, so we were not able to hypothesize why the direct method was incompatible. Although our data suggest that the inhibitory factor(s) were site specific–not patient specific–prior studies have also suggested that mucosal material can contribute to signal inhibition [18]. We recommend that laboratories seeking to employ the direct method for SARS-CoV-2 detection should conduct a small pilot run (comparing results from direct and full PCR analyses on the same samples) to ensure that sample media are compatible with this method. This pilot should be replicated if/when sample collection methods or media are changed.

The success of this ring trial is of critical importance given the growing calls for COVID-19 screening as a containment strategy. The growing pandemic requires that we supplement definitive clinical testing with scalable screening strategies that generate efficient, reliable results that can readily inform public health action (e.g., quarantine and isolation) [2]. Non-PCR immunoassay antigen screening kits have decreased sensitivity as compared to standard PCR but are widely utilized depending on the country’s COVID-19 pandemic testing strategy [19]. As anticipated per previous studies, the direct method as applied to SARS-CoV-2 results in some loss in sensitivity compared to standard PCR. One primary explanation for this observation is that RNA extraction typically concentrates RNA present in the clinical sample (by eluting the sample in a smaller volume. In addition, there is a low level of inhibition seen in clinical NP samples loaded directly into an RT-PCR reaction, and the sensitivity of the approach drops when more than 3 ul of patient sample is used [10]. However, this loss is of lower significance to the method’s potential value as a public health screening tool. The direct method succeeds in all of the areas of greatest contemporary need: 1) it reliably detects samples with RNA levels correlating to the presence of live virus (and thus most potential for infectivity), 2) it provides the potential to optimize throughput and reduce costs/logistics for SARS-CoV-2 testing, 3) it is a methodology with no commercial barriers or de novo equipment investment hurdles, and 4) it can be readily adopted by most current public health or clinical laboratories with experience handling infectious samples [14,15].

We believe that the direct RT-qPCR method for SARS-CoV-2 screening is ripe for adoption in laboratories seeking to reduce turnaround time for processing samples, experiencing challenges in accessing extraction reagents, seeking to decrease costs, and/or looking to reduce the use, handling, and disposal of chemicals in their laboratory. We do not propose this method as a substitute for samples requiring ultrasensitive detection. As with the adoption of any new method, appropriate validation must be conducted by the host laboratory. As standard RNA extraction reagents for PCR can cost $5–$6 USD per extraction and millions of these tests are performed each day around the world, the potential savings are significant. The utilization of this method could lead to greater testing coverage of individuals per dollar invested, or alternatively a larger number of examinations per individual, either of which would allow for the follow up of suspected cases.

The opportunity and feasibility described here is not simply theoretical. At the time of publication, several of the Propagate Network partner laboratories (Brazil, France, United States) are promoting or exploring the broad-scale adoption and implementation of this method for ongoing SARS-CoV-2 public health screening efforts in their regions [20]. Additionally, in October 2020, the Infectious Disease Diagnostic Laboratory at the Children’s Hospital of Philadelphia implemented an extraction-free protocol for routine diagnostic testing of SARS-CoV-2 [21]. In the 3 months following implementation, >40,000 samples were tested using this workflow. The laboratory observed several critical advantages with this approach, including dramatically reduced extraction reagent costs and a halving of the average laboratory turnaround time, despite increasing test volumes. Further, the independence from specialized extraction reagents for routine testing alleviated pressure on supply chains to meet the increased demand. These same positive impacts on testing efficiency are expected to apply to other laboratories that adopt the method.

While no current testing or screening method is optimal to all situations, the direct method should be considered as a viable, fit-for-purpose resource to address the growing need for population monitoring during a challenging vaccination rollout and amidst the emergence of increasingly virulent strains of SARS-CoV-2.

In addition to the valuable data described above, the global viral testing network established for this study exemplifies the feasibility and importance of establishing transparent, transparent and accessible engagements in the public health sciences. Following this study, the Propagate Network will continue to serve as a forum for scientific information exchange and collaboration in the face of future pandemics or health challenges.

5. Conclusions

The need for testing for SARS-CoV-2 continues and in many regions is increasing dramatically. This study provides multisite evidence that the direct RT-PCR method can be employed for the detection of SARS-CoV-2 viral RNA with the omission of the RNA extraction step and its associated extraction reagents. This effort represents the first step toward simplifying detection of SARS-CoV-2 viral RNA for the global research community by leveraging evidence-based guidance such as the results present herein. Many options for detecting SARS-CoV-2 have emerged recently, such as antibody testing, saliva testing, and point-of-care testing, which taken together support the urgent need for actionable viral testing. This work lays the foundation for an adoptable method for future viral outbreaks.

Acknowledgments

The authors thank and acknowledge Vianney Leclercq (LBM BIOESTEREL, Mouans-Sartoux, France) for assistance in sample analysis and Bob Bruneau (University of Washington, USA) for coordinating international shipping of samples and reagents.

EPA Disclaimer

The views expressed in this article are those of the author(s) and do not necessarily represent the views or the policies of the U.S. Environmental Protection Agency.

References

  1. 1. World Health Organization. COVID-19 strategy update Geneva, Switzerland: WHO; 2020 [24 March 2021]. Available from: https://www.who.int/publications/m/item/covid-19-strategy-update.
  2. 2. Mina MJ, Parker R, Larremore DB. Rethinking COVID-19 test sensitivity—a strategy for containment. N Engl J Med. 2020;383(22):e120. Epub 2020/10/01. pmid:32997903.
  3. 3. Pettit SD, Jerome KR, Rouquié D, Mari B, Barbry P, Kanda Y, et al. ’All In’: a pragmatic framework for COVID-19 testing and action on a global scale. EMBO Mol Med. 2020;12(6):e12634. Epub 2020/05/07. pmid:32375201; PubMed Central PMCID: PMC7267598 reflect the views or policies of the U.S. Environmental Protection Agency, Japan National Institute of Health Sciences, the Pharmaceuticals and Medical Devices Agency, or that of any of the authors’ employers. The authors have no conflicts to disclose.
  4. 4. World Health Organization. Laboratory testing for coronavirus disease 2019 (COVID-19) in suspected human cases. Interim guidance 2 March 2020 Geneva, Switzerland: WHO; 2020 [24 March 2021]. Available from: https://apps.who.int/iris/handle/10665/331329.
  5. 5. Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020;25(3):2000045. Epub 2020/01/30. pmid:31992387; PubMed Central PMCID: PMC6988269.
  6. 6. U.S. Centers for Disease Control and Prevention. CDC 2019-novel coronavirus (2019-nCoV) real-time RT-PCR diagnostic panel for emergency use only: instructions for use 2020 [24 March 2021]. Available from: https://www.fda.gov/media/134922/download.
  7. 7. American Society for Microbiology. Supply shortages impacting COVID-19 and non-COVID testing 2021 [24 March 2021]. Available from: https://asm.org/Articles/2020/September/Clinical-Microbiology-Supply-Shortage-Collecti-1.
  8. 8. Millan C, Navarro A, Kueffner S. Test scarcity means Latin America is ‘walking blindly’ on virus 2020 [24 March 2021]. Available from: https://www.bloomberg.com/news/articles/2020-04-08/test-scarcity-makes-tracking-virus-in-latin-america-a-crapshoot.
  9. 9. Soy A. Lack of Covid-19 testing undermines Africa’s ’success’ 2020 [24 March 2021]. Available from: https://www.bbc.com/news/world-africa-52801190.
  10. 10. Bruce EA, Huang ML, Perchetti GA, Tighe S, Laaguiby P, Hoffman JJ, et al. Direct RT-qPCR detection of SARS-CoV-2 RNA from patient nasopharyngeal swabs without an RNA extraction step. PLoS Biol. 2020;18(10):e3000896. Epub 2020/10/03. pmid:33006983; PubMed Central PMCID: PMC7556528 competing interests. While DJS was employed at IXIS LLC at the time of this study, his employment there did not create a competing interest. Further, IXIS LLC had no involvement in this study.
  11. 11. Smyrlaki I, Ekman M, Lentini A, Rufino de Sousa N, Papanicolaou N, Vondracek M, et al. Massive and rapid COVID-19 testing is feasible by extraction-free SARS-CoV-2 RT-PCR. Nat Commun. 2020;11(1):4812. Epub 2020/09/25. pmid:32968075; PubMed Central PMCID: PMC7511968.
  12. 12. Arons MM, Hatfield KM, Reddy SC, Kimball A, James A, Jacobs JR, et al. Presymptomatic SARS-CoV-2 Infections and transmission in a skilled nursing facility. N Engl J Med. 2020;382(22):2081–90. Epub 2020/04/25. pmid:32329971; PubMed Central PMCID: PMC7200056.
  13. 13. La Scola B, Le Bideau M, Andreani J, Hoang VT, Grimaldier C, Colson P, et al. Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards. Eur J Clin Microbiol Infect Dis. 2020;39(6):1059–61. Epub 2020/04/29. pmid:32342252; PubMed Central PMCID: PMC7185831.
  14. 14. van Kampen JJA, van de Vijver DAMC, Fraaij PLA, Haagmans BL, Lamers MM, Okba N, et al. Shedding of infectious virus in hospitalized patients with coronavirus disease-2019 (COVID-19): duration and key determinants. medRxiv [Internet]. 2020 24 March 2021:[2020.06.08.20125310 p.]. Available from: http://medrxiv.org/content/early/2020/06/09/2020.06.08.20125310.abstract.
  15. 15. Wölfel R, Corman VM, Guggemos W, Seilmaier M, Zange S, Müller MA, et al. Virological assessment of hospitalized patients with COVID-2019. Nature. 2020;581(7809):465–9. Epub 2020/04/03. pmid:32235945.
  16. 16. Graham NR, Whitaker AN, Strother CA, Miles AK, Grier D, McElvany BD, et al. Kinetics and isotype assessment of antibodies targeting the spike protein receptor-binding domain of severe acute respiratory syndrome-coronavirus-2 in COVID-19 patients as a function of age, biological sex and disease severity. Clin Transl Immunology. 2020;9(10):e1189. Epub 2020/10/20. pmid:33072323; PubMed Central PMCID: PMC7541824.
  17. 17. Kirkland PD, Frost MJ. The impact of viral transport media on PCR assay results for the detection of nucleic acid from SARS-CoV-2. Pathology. 2020;52(7):811–4. Epub 2020/12/01. pmid:33250079; PubMed Central PMCID: PMC7534658.
  18. 18. Kuiper JWP, Baade T, Kremer M, Kranaster R, Irmisch L, Schuchmann M, et al. Detection of SARS-CoV-2 from raw patient samples by coupled high temperature reverse transcription and amplification. PLOS ONE. 2020;15(11):e0241740. pmid:33137168
  19. 19. Eshghifar N, Busheri A, Shrestha R, Beqaj S. Evaluation of analytical performance of seven rapid antigen detection kits for detection of SARS-CoV-2 virus. Int J Gen Med. 2021;14:435–40. Epub 2021/02/20. pmid:33603450; PubMed Central PMCID: PMC7886288.
  20. 20. Fassy J, Lacoux C, Leroy S, Noussair L, Hubac S, Degoutte A, et al. Versatile and flexible microfluidic qPCR test for high-throughput SARS-CoV-2 and cellular response detection in nasopharyngeal swab samples. medRxiv [Internet]. 2020 24 March 2021:[2020.11.09.20228437 p.]. Available from: https://www.medrxiv.org/content/10.1101/2020.11.09.20228437v1. pmid:32511587
  21. 21. Dumm RE, Elkan M, Fink J, Richard-Greenblatt M, Obstfeld AE, Harris RM. Implementation of an Extraction-Free COVID Real-Time PCR Workflow in a Pediatric Hospital Setting. J Appl Lab Med. 2021;6(6):1441–51. Epub 2021/06/25. pmid:34165533; PubMed Central PMCID: PMC8394822.