https://orcid.org/0000-0001-5186-5600
Figures
Abstract
Background
COVID-19 is an ongoing pandemic leading to exhaustion of the hospital care system. Our health care system has to deal with a high level of sick leave of health care workers (HCWs) with COVID-19 related complaints, in whom an infection with SARS-CoV-2 has to be ruled out before they can return back to work. The aim of the present study is to investigate if the recently described CoLab-algorithm can be used to exclude COVID-19 in a screening setting of HCWs.
Methods
In the period from January 2021 till March 2021, HCWs with COVID-19-related complaints were prospectively collected and included in this study. Next to the routinely performed SARS-CoV-2 RT-PCR, using a set of naso- and oropharyngeal swab samples, two blood tubes (one EDTA- and one heparin-tube) were drawn for analysing the 10 laboratory parameters required for running the CoLab-algorithm.
Results
In total, 726 HCWs with a complete CoLab-laboratory panel were included in this study. In this group, 684 HCWs were tested SARS-CoV-2 RT-PCR negative and 42 cases RT-PCR positive. ROC curve analysis showed an area under the curve (AUC) of 0.853 (95% CI: 0.801–0.904). At a safe cut-off value for excluding COVID-19 of -6.525, the sensitivity was 100% with a specificity of 34% (95% CI: 21 to 49%). No SARS-CoV-2 RT-PCR cases were missed with this cut-off and COVID-19 could be safely ruled out in more than one third of HCWs.
Conclusion
The CoLab-score is an easy and reliable algorithm that can be used for screening HCWs with COVID-19 related complaints. A major advantage of this approach is that the results of the score are available within 1 hour after collecting the samples. This results in a faster return to labour process of a large part of the COVID-19 negative HCWs (34%), next to a reduction in RT-PCR tests (reagents and labour costs) that can be saved.
Citation: Leers MPG, Deneer R, Mostard GJM, Mostard RLM, Boer A-K, Scharnhorst V, et al. (2022) Use of an algorithm based on routine blood laboratory tests to exclude COVID-19 in a screening-setting of healthcare workers. PLoS ONE 17(6): e0270548. https://doi.org/10.1371/journal.pone.0270548
Editor: Siddhartha Pratim Chakrabarty, Indian Institute of Technology Guwahati, INDIA
Received: June 4, 2021; Accepted: June 13, 2022; Published: June 28, 2022
Copyright: © 2022 Leers et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Due to legal restrictions on sharing the de-identified data, individual level data cannot be shared publicly. However, these data will be made available to individuals who want access for scientific and/or academic research purposes and are willing to commit to handling the data in a manner which is consistent with confidentiality requirements. The criteria for access would broadly incorporate requests from individuals with credible academic/research credentials. Requests for access to the data should be directed to the Ethics Committee of Zuyderland Medical Centre at METC@zuyderland.nl.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) disease 2019 (COVID-19)) is an ongoing pandemic with at present over 150 million of cases and over three million deaths worldwide [1]. The initial clinical symptoms for COVID-19 are nonspecific and similar to other seasonal viral diseases, which encompass fever, dyspnoea, dry cough and fatigue. Many countries, including the Netherlands, are struggling to control COVID-19 outbreaks, especially in the detection of silent infections in the pre- or asymptomatic patient that can contribute to transmission [2]. Empirical studies have indicated that individuals may be highly infectious during the presymptomatic phase [3].
Healthcare workers (HCWs) potentially experience greater risks for emerging infectious diseases [4,5] due to occupational exposure to sick patients and virus-contaminated surfaces [6]. Contagious HCWs may infect patients, co-workers and family members. However, the withdrawal of ill HCWs from duty can threaten essential healthcare staffing during an epidemic [7]. Therefore, infection prevention and quick, accurate diagnosis of potential COVID-19 in HCWs are crucial to maintain hospital operations [8].
Consequently, understanding the prevalence of, and factors associated with SARS-CoV-2 infection among frontline HCWs who care for COVID-19 patients are important to protect both HCWs and their patients. Next to this, modelling analyses show that rapid case identification of infected persons is critical to interrupt transmission, especially for infectious cases without clinical symptoms [2]. At the moment, the health care system has to deal with a high level of absenteeism of HCWs with COVID-19 related complaints, and in whom an infection with SARS-CoV-2 has to be ruled out before they can return back to work.
Reverse Transcription polymerase chain reaction (RT-PCR) based methodologies are the gold standard in confirming that the individual presenting with COVID-19 has active viral shedding of SARS-CoV-2 [9]. However, there are some important limitations to RT-PCR. First, current techniques take up to 6–8 hours in order to obtain first results. Next to this, laboratories often cannot handle the overload of tests. A third important limitation is that RT-PCR on a nasopharyngeal swab, may be false negative in the initial phase of the disease, in spite of the presence of typical symptoms [10–12]. In addition, the standard test used has an 80% accuracy (compared to chest CT scan results) [12], which may depend on the specific level of viral shedding by any individual at the time of sample test. Fourth, the RT-PCR technique carries a certain cost, which could mean a considerable financial burden [13].
Very recently, an algorithm was developed by Boer et al, leading to the so-called ‘CoLab’-score [14]. The score is calculated using 10 numeric values of routine-laboratory parameters next to the age of the patient. The linear predictor of the CoLab-score is continuous, therefore a cut-off can be chosen such that a high sensitivity and high negative predictive value can be achieved. This algorithm was developed and validated to exclude COVID-19 in patients presenting at the Emergency Department (ED).
The aim of the present study was to investigate if the CoLab-score could be used to exclude within one hour COVID-19 in a screening setting of healthcare workers, who requested a SARS-CoV-2 RT-PCR test because of COVID-19 related complaints, or because they were in close proximity to a SARS-CoV-2 infected person.
Materials & methods
Study design and selection of healthcare workers (HCWs)
We conducted a prospective screening study to assess the comparability between naso-/oropharyngeal swabs and the CoLab-score (based on routine blood tests). Healthcare workers were included during the period from January 2021 till March 2021 either:
- because of COVID-19 related complaints or
- because they were in close proximity to a person with COVID-19.
HCWs were required to have complete data on clinical chemistry and hematologic parameters, needed to calculate the CoLab-score. From the validation study [14] it is known that there are some external factors influencing the predictive value of the score. For this reason, the following HCWs were excluded from this study:
HCWs with:
- more than 10 days complaints at the time of screening
- a known positive SARS-CoV-2 RT PCR in the past 4 weeks
- invalid RT-PCR test results due to contamination
Next to this, the data of the following HCWs were also excluded because of known interference with the algorithm [14]:
- a deep anemia at the time of presentation to the ED (Hb < 5.5 mmol/L)
- extreme laboratory values (>10* standard deviation (SD)) in one or more of the Colab-values)
Using a standard protocol, paired naso-/oropharyngeal swabs from HCWs were collected using sterile flocked E-swabs and placed both in one sterile tube containing viral transport medium. Next to this, blood was collected in heparin- and EDTA-anticoagulated blood containers. The samples were transported to the central laboratory and immediately prepared for analysis.
Laboratory measurements
For clinical chemistry and hemocytometric analyses, heparin- and EDTA-anticoagulated venous blood samples respectively were collected. All analyses were performed at presentation. Clinical chemistry parameters (c-reactive protein (CRP), albumin, total bilirubin, alkaline phosphatase (AP), γ-glutamyl transferase (GGT) and lactate dehydrogenase (LDH)) were obtained on routine chemistry analysers from Roche (Cobas; Roche Diagnostics, Basel, Switzerland). The hemocytometric parameters (leukocytes, erythrocytes, eosinophilic and basophilic granulocytes) were derived from a complete blood count (CBC) measured on a XN-1000 (Sysmex, Kobe, Japan).
The nasopharyngeal and oropharyngeal swab samples were obtained for SARS-CoV-2 detection using multiplex Real-Time Polymerase Chain Reaction (RT-PCR) using QIAsymphony DSP Virus/ Pathogen Mini Detection Kit (Qiagen Inc., https://www.qiagen.com). Both the binary outcome of the RT-PCR (positive or negative), as well as the cycling time-value (Ct-value) in case of a positive RT_PCR were registered.
In case of a negative RT-PCR result in a HCW with persistent high suspicion for COVID-19 (e.g. suggestive symptoms without apparent alternative cause) the RT-PCR test could be repeated after 48 hr of the initial RT-PCR. The exclusion criterium of no more than 10 days complaints at the time of screening then still applies.
CoLab-score calculation
The Colab-score is described in detail elsewhere [14]. In short, it is calculated by plugging the ten obtained laboratory measurands, next to the age of the HCW, into a formula: (– 6.885 + [erythrocytes (*1012/L)] × 0.9379 –[leukocytes (*109/L)] × 0.1298 –[eosinophils (*109/L)] × 6.834 –[basophils (*109/L)] × 47.7 –log10([bilirubin (μmol/L)]) × 1.142 + log10([LDH (U/L)]) × 5.369 –log10([AP (U/L)]) × 3.114 + log10([γGT (U/L)]) × 0.3605 –[albumin (g/L)] × 0.1156 + [CRP (mg/L)] × 0.02560 + [age (years, 2 decimals)] × 0.002275). This results in a numeric value, called the CoLab-linear predictor. This linear predictor can be converted to a score using the cut-offs described in the original publication [14].
Statistical analysis
Since the CoLab-score was developed to screen patients presenting at the emergency department (ED) for a possible COVID-19 infection, rather than exclude a SARS-CoV-2 infection in HCWs, the suitability for screening HCWs was investigated in this study. First, the discriminative ability of the CoLab-linear predictor was assessed by calculating the area under the ROC curve (AUC). Secondly, model calibration was visually assessed with a calibration plot where the CoLab-linear predictor was converted to the predicted probability (through the inverse logit function) and the proportion of observed outcomes was plotted versus expected probabilities [15]. A logistic regression model was fitted to the CoLab predicted probabilities to assess model calibration in terms of intercept and slope [15]. This was done by plotting the proportion of observed COVID-19 positives versus expexted probabilities. Ideally, observed proportions are equal to expected proportions, and this ideal-calibration line is shown as a straight line through the origin with a slope of 1. The logistic calibration line will be a logistic regression fit of the predicted probabilities. Using the intercept and/or slope from the logistic regression model, recalibrated probabilities were obtained and also plotted in a calibration plot. Thirdly, a cut-off for the CoLab-linear predictor was calculated to safely rule-out a COVID-19 infection in HCWs with an estimated 95% sensitivity. This was done by fitting a Gaussian to the distribution of the CoLab-linear predictor for all HCWs tested positive for SARS-CoV-2. The cut-off to safely rule out COVID-19 was chosen as the 5th percentile of the fitted Gaussian distribution. The number needed to screen (defined as the number of HCWs needed to RT-PCR test to find one positive) is calculated by dividing the total number of HCWs below the cut-off by the number of HCWs above the cut-off and tested RT-PCR positive. The fraction of HCWs falling below the cut-off was calculated to determine the potential reduction in RT-PCR tests. Confidence intervals for the Gaussian fit, 5th percentile and potential reduction in RT-PCR tests were obtained by bootstrapping and calculating the bias-corrected and accelerated bootstrap (BCa) confidence intervals (CIs). Finally, the relation between the RT-PCR CT-value and CoLab-linear predictor was plotted to determine if higher CT-values corresponded to lower CoLab-linear predictor values. All statistical analyses were performed in R version 4.0.5 [16], calibration plots were made using the rms-package [17], bootstrapping was done using the boot-package [18].
Ethical considerations
The medical ethics committee of ZMC (Zuyderland METC Zuyd, registration nr. METCZ2021002) approved this study. Data were acquired after informed consent and obtained in accordance with the Declaration of Helsinki, version 2013. Participation in this study was voluntary, and each participating HCW obtained a hard copy of the ‘test subject information sheet’, in which the study is explained and were the participant has to give written consent. These signed consent forms were also signed by the study personel member responsible for the venipuncture. Each participant was also aware that they could opt out at any time. Because the study was restricted to HCWs, no minors (<18 years) were included.
Results
In total, 775 health care workers (HCWs) were included in this study. Forty-nine out of the 775 HCWs were excluded (Fig 1). A total of 42 HCWs (5.8%) were SARS-CoV-2 RT-PCR positive.
Descriptive statistics for the 726 included HCWs, grouped by RT-PCR test result, are shown in Table 1. Age, erythrocytes, LD, AP and γGT do not show significant differences between RT-PCR positive and RT-PCR negative groups. All other variables included in the CoLab-score differ significantly between the 2 groups.
Shown are the laboratory tests required for the CoLab-score and their mean/median results split by RT-PCR test result. For results with normal distributions, the mean value and standard deviation (in round brackets) are shown. For results that have skewed or heavy tailed distributions, the median value and the interquartile range is shown [in squared brackets]. The p-value corresponds to a t-test in cases of a normal distribution, a Man-Whitney U-test for non-normally distributed variables and a Fisher exact test for categorical variables.
ROC-curve analysis of the CoLab-linear predictor is shown in Fig 2. The AUC of the CoLab-linear predictor in discriminating between RT-PCR positive and negative HCWs was 0.853 (95% CI: 0.801–0.904).
The area under the ROC curve is shown with the 95% DeLong confidence interval in round brackets. The displayed threshold of -6.241 corresponds to a sensitivity of 100%, i.e. no HCWs below this linear predictor were RT-PCR positive.
The calibration plot corresponding to the predicted probabilities and observed proportion of RT-PCR positives is plotted in Fig 3A. The logistic regression calibration slope is equal to 1.056 (SE: 0.1438) and the intercept 2.322 (SE: 0.6197). This implies that predicted probabilities are systematically too low but re-calibration is straightforward as there is no evidence that the slope is ≠ 1, hence only the intercept term needs to be added to the original CoLab-linear predictor to obtain a re-calibrated linear predictor suitable for screening HCWs. The re-calibrated calibration plot is show in Fig 3B. This also illustrates that the discriminative ability of the CoLab-linear predictor is preserved but that thresholds for screening HCWs should be lower than ED patients.
A. In the calibration plot the proportion of observed COVID-19 positives versus expected proportion of positives are plotted. Observations are grouped with an average of 50 observations per group. The expected probabilities follow from applying the inverse logit function to the CoLab-linear predictor. If the observed proportion in an external dataset is lower than the expected proportion, this means risks are over-estimated, if the observed fraction is higher, risks are under-estimated. Ideally, observed proportions are equal to expected proportions, this ideal-calibration-line is shown as a straight line through the origin with a slope of 1. The logistic calibration line is a logistic regression fit of the predicted probabilities. B. Using the intercept and/or slope from the logistic regression model, recalibrated probabilities were obtained and plotted in a second calibration plot.
To define a safe cut-off for excluding COVID-19 in HCWs, a Gaussian is fitted to the distribution of CoLab-linear predictor of HCWs that were tested RT-PCR positive (Fig 4). The Shapiro-Wilk test showed no evidence of non-normality (P-value = 0.621). The 5th percentile of the Gaussian fit of the CoLab-linear predictor is equal to -6.525 (95% CI: -7.147 to -5.999), which is recommended as the cut-off below which COVID-19 can be safely ruled-out in HCWs. Using the -6.525 cut-off, the percentage of HCWs that can be safely excluded is 34% (95% CI: 21 to 49%), with a specificity of 34%, a sensitivity of 100%, a positive predictive value of 9% and a negative predictive value of 100%. The number need to test is 12 (95% CI: 10 to 14).
A normal distribution was fitted to the RT-PCR negative group (mean: -6.04, SD: 1.73), the dashed lines represent the 95% CI. The 5th percentile of the Gaussian distribution is shown in red and dashed lines represent the 95% CI. Linear predictor values below this 5th percentile are regarded as non-COVID-19.
In Fig 5 the relationship between the CoLab-linear predictor and the RT-PCR CT value is plotted. The fitted smooth in Fig 5 shows a rising CT value (implying a decreasing amount of template) near the lower end of the CoLab-linear predictor.
The CoLab-linear predictor is plotted versus the RT-PCR CT value. The red line is the CoLab-linear predictor cut-off below which HCWs are regarded as non-COVDI 19, the dashed red lines represent the 95% CI of the cut-off. The dashed line is a LOESS smooth where the 95% confidence interval is shown in gray.
Discussion
In this prospective study among healthcare workers (HCWs), it was shown that the model behind the CoLab-score could be used to safely exclude COVID-19 in HCWs. The original cut-off for the CoLab-linear predictor was adapted for excluding COVID-19 in HCWs. Using this adapted cut-off, a NPV of 100% was found with a specificity of 34% (95% CI: 21 to 49%). The number needed to screen by using the CoLab guided SARS-CoV-2 RT-PCR testing was 12 (95% CI: 10 to 14). The overall prevalence of COVID-19 in this group of HCWs was 5.8%. In 33 cases (4.3%) the CoLab score could not be calculated. This was due to haemolysis of the blood sample, caused by an improper venipuncture.
RT-PCR-based methodologies are the gold standard for confirming COVID-19. There are several factors that can contribute to false-negative results, including the adequacy of the specimen collection technique, time from exposure and specimen source. Furthermore, current and future viral changes could affect viral based diagnostics [19–21]. In addition, several studies have already shown that COVID-19 is characterized by biochemical as well as haematological changes in peripheral blood [22–25]. Next to focusing on the viral response (RT-PCR), investigating the host immune response by analysing biochemical and haematological changes in peripheral blood is an attractive alternative method [26]. As shown in the living systematic review from Wynants et al [27], a considerable number of prediction models for COVID-19 have been published until recently, and biochemical and haematological parameters are often an important part of these prediction models.
Recently the CoLab-score was developed and externally validated by Boer et al [14] to exclude COVID-19 in patients presenting at the Emergency Department, using an adaptive LASSO-regression technique [28]. This score is based on 6 biochemical and 4 haematological parameters, next to the age of the patient. It appears that the strength of the high NPV derived from this algorithm is driven by the absence of specific COVID-19 related biochemical and haematological changes in peripheral blood.
As the CoLab-score is based on a categorization of the underlying continuous linear predictor, the cut-offs define the diagnostic properties of the individual scores. Our results show that the discriminative ability of the CoLab-linear predictor is preserved when screening HCWs instead of patients presenting at the emergency department, as indicated by the AUC and calibration slope. The AUC is lower than the development group of the original CoLab study [14], but similar to the AUC reported for external datasets in the original CoLab study. Therefore, the discriminative ability seems to be preserved when classifying HCWs.
The cut-offs defined in the original CoLab publication are however not suitable for excluding COVID-19 in HCWs. This is confirmed by the calibration intercept which shows that probabilities predicted by the original CoLab-linear predictor are systematically too low for HCWs. As reliable exclusion of COVID-19 is of utmost importance in screening HCWs, a logical choice would be to select the highest cut-off with 100% sensitivity. However, one might speculate that when sampling more HCWs, the sensitivity could drop, and RT-PCR positive HCWs could occur even below this threshold. Therefore, the Gaussian distribution was fitted to the data and the 5th percentile chosen as safe cut-off. Doing so, the reliability increases at the expense of the number of “negative” results. We recommend that the optimal cut-off value is -6.525, where COVID-19 could be excluded in about 34% of the HCW’s. Furthermore, Fig 5 suggests that potentially “missed” COVID-19 HCWs might have relative high CT values, potentially resulting in lower disease burden and contagiousness. It must be kept in mind that this study was performed in a time that the prevalence of COVID-19 was high (>10%) and no one want to classify a possible COVID-19 positive HCW as false-negative. For that reason, the optimal cut-off concerns now a very save cut-off. It can be hypothesized that when COVID-19 prevalence drops, the cut-off value can be adapted by investigating which negative predictive value can be allowed in this new setting.
It turned out that the cut-off to be used in the CoLab-algorithm in the HCW screening setting is different from that of patients presenting at the ED. This could be explained by the fact that in the screening setting the duration of the infective period is shorter, the complaints are milder and consequently the host-immune response also less pronounced [26]. In addition, in our study group there were 2 HCWs in which the SARS-CoV-2 RT-PCR was initially negative, while the CoLab-score was not negative. A week later both HCWs had a retest because of persistent COVID-19 related complaints. At that time the outcome of the linear predictor had worsened, and at that time the repeated RT-PCR test turned out to be positive. Because these 2 HCWs had more than 10 days COVID-19-related complaints, they were excluded from this study. It seems that the outcome of the CoLab-score is dynamic and follows the host immune response.
At this time, also so-called lateral flow tests (LFTs) are available which detects the presence of SARS-COV-2 antigen. They are widely adopted In ED’s because of their ease of use and the rapid result (<30 minutes). However, compared to these LFTs which provide a dichotomous results, the CoLab-score provides a continuous score. Using the above-mentioned cut-off value, the CoLab-score offers a higher sensitivity and are therefore more suitable to rule-out COVID-19 than a LFT, which are only moderately sensitive [29,30].
A limitations of this study is that this study has been performed in a period where only the original SARS-CoV-2 next to only the alpha-variant of it were present. Since the CoLab-score reflects the host-response to the virus, it is expected that the accuracy of the score will not be changed by emerging SARS-CoV-2 variants. This assertion is supported by Boer et al, how found sustained diagnostic performance of the CoLab-score in periods with different dominant variants (especially Alpha- and Delta-variant) [14].
A control group of HCWs who do not have complaints or have been in close proximity of a COVID-19 patient was ideally a good control that could assist Iin the understanding for the presence of any possible that could produce false-positive results. Unfortunately, this was practically not feasible because the study was designed and performed in a period with high absenteeism of HCWs due to COVID-19. The medical board as well as the ethical committee of our hospital found it unethically to test HCWs with no complaints, especially with the knowledge that a positive SARS-CoV-2 RT-PCR test result does not always mean that the person can spread the virus actively. Because the focus of our study was directed to develop a screening method with a high negative predictive value, the missing of this control group can be remarked as a limitation of our study.
Next to this, these HCWs concerns a group of possible patients that are not registered in our hospital information system, and are treated completely anonymous for us. For that reason, we don’t have information about the kind of work the HCW perfor (e.g. nurse, physician, laboratory personnel, cleaning staff, transport) and for that reason it is not possible to assess the group of HCWs who work in close proximity of a COVID-19 patient, separately from the group of HCWs who don’t work in close proximity.
In conclusion, the CoLab-score is an easy and reliable algorithm that, using an adapted cut-off, can be used in screening HCWs with COVID-19 related complaints. Major advantages of this approach are that the results of the score are available within 1 hour after collecting the samples, it can be implemented in almost every hospital, even in a 24/7 setting, and the costs are minimal when compared to RT-PCR. This results in a faster return to labour process of a significant part of the COVID-19 negative HCWs (34%), next to a reduction in RT-PCR tests (reagents and labour duties) that can be saved.
Acknowledgments
The authors would like to thank Christel Jacquot, Anke Linssen and Audrey Merry for their help with this study. Additionally, authors would like to thank the laboratory technician René Nieuwenhuysen for extracting the data out of the laboratory information systems. Finally, we appreciate the valuable assistance of the phlebotomists of the Department of Clinical Chemistry & Hematology of Zuyderland MC, Heerlen/Sittard-Geleen.
References
- 1. World Health Organization WHO. COVID-19 weekly epidemiological update. Novel coronavirus (2019-nCoV) situation reports 2021. 2021.
- 2. Moghadas SM, Fitzpatrick MC, Sah P, Pandey A, Shoukat A, Singer BH, et al. The implications of silent transmission for the control of COVID-19 outbreaks. Proc Natl Acad Sci U S A. 2020;117(30):17513–5. pmid:32632012
- 3. He X, Lau EHY, Wu P, Deng X, Wang J, Hao X, et al. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nat Med. 2020;26(5):672–5. pmid:32296168
- 4. Lan FY, Wei CF, Hsu YT, Christiani DC, Kales SN. Work-related COVID-19 transmission in six Asian countries/areas: A follow-up study. PLoS One. 2020;15(5):e0233588. pmid:32428031
- 5. Offeddu V, Yung CF, Low MSF, Tam CC. Effectiveness of Masks and Respirators Against Respiratory Infections in Healthcare Workers: A Systematic Review and Meta-Analysis. Clin Infect Dis. 2017;65(11):1934–42. pmid:29140516
- 6. Reusken CB, Buiting A, Bleeker-Rovers C, Diederen B, Hooiveld M, Friesema I, et al. Rapid assessment of regional SARS-CoV-2 community transmission through a convenience sample of healthcare workers, the Netherlands, March 2020. Euro Surveill. 2020;25(12). pmid:32234115
- 7. Fraher EP, Pittman P, Frogner BK, Spetz J, Moore J, Beck AJ, et al. Ensuring and Sustaining a Pandemic Workforce. N Engl J Med. 2020;382(23):2181–3. pmid:32268021
- 8. Rueda-Garrido JC, Vicente-Herrero MT, Del Campo MT, Reinoso-Barbero L, de la Hoz RE, Delclos GL, et al. Return to work guidelines for the COVID-19 pandemic. Occup Med (Lond). 2020;70(5):300–5. pmid:32476022
- 9. van Kampen JJA, van de Vijver D, Fraaij PLA, Haagmans BL, Lamers MM, Okba N, et al. Duration and key determinants of infectious virus shedding in hospitalized patients with coronavirus disease-2019 (COVID-19). Nat Commun. 2021;12(1):267. pmid:33431879
- 10. Guo L, Ren L, Yang S, Xiao M, Chang, Yang F, et al. Profiling Early Humoral Response to Diagnose Novel Coronavirus Disease (COVID-19). Clin Infect Dis. 2020;71(15):778–85. pmid:32198501
- 11. Li Q, Guan X, Wu P, Wang X, Zhou L, Tong Y, et al. Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus-Infected Pneumonia. N Engl J Med. 2020;382(13):1199–207. pmid:31995857
- 12. Lippi G, Simundic AM, Plebani M. Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19). Clin Chem Lab Med. 2020;58(7):1070–6. pmid:32172228
- 13. Langer T, Favarato M, Giudici R, Bassi G, Garberi R, Villa F, et al. Development of machine learning models to predict RT-PCR results for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in patients with influenza-like symptoms using only basic clinical data. Scand J Trauma Resusc Emerg Med. 2020;28(1):113. pmid:33261629
- 14. Boer A-K, Deneer R, Maas M, Ammerlaan HSM, Balkom van RHH, Leers MPG, et al. The CoLab-score rapidly and efficiently excludes COVID-19 at the Emergency Department without additional testing. BMJ Open 2022: accepted.
- 15. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38. pmid:20010215
- 16. R Core Team. R: A Language and Environment for Statistical Computing. 2020.
- 17. Harrel FE. RMS: Regression Modelling Strategies. R package version 6.2–0 ed2021.
- 18. Canty A, Ripley B. Boot: Bootstrap R (S-Plus) functions. R package version 1–3.28 ed2021.
- 19. Artesi M, Bontems S, Gobbels P, Franckh M, Maes P, Boreux R, et al. A Recurrent Mutation at Position 26340 of SARS-CoV-2 Is Associated with Failure of the E Gene Quantitative Reverse Transcription-PCR Utilized in a Commercial Dual-Target Diagnostic Assay. J Clin Microbiol. 2020;58(10).
- 20. Zhou Y, Pei F, Ji M, Wang L, Zhao H, Li H, et al. Sensitivity evaluation of 2019 novel coronavirus (SARS-CoV-2) RT-PCR detection kits and strategy to reduce false negative. PLoS One. 2020;15(11):e0241469. pmid:33206690
- 21. Ziegler K, Steininger P, Ziegler R, Steinmann J, Korn K, Ensser A. SARS-CoV-2 samples may escape detection because of a single point mutation in the N gene. Euro Surveill. 2020;25(39). pmid:33006300
- 22. Kurstjens S, van der Horst A, Herpers R, Geerits MWL, Kluiters-de Hingh YCM, Gottgens EL, et al. Rapid identification of SARS-CoV-2-infected patients at the emergency department using routine testing. Clin Chem Lab Med. 2020;58(9):1587–93. pmid:32598302
- 23. Linssen J, Ermens A, Berrevoets M, Seghezzi M, Previtali G, van der Sar-van der Brugge S, et al. A novel haemocytometric COVID-19 prognostic score developed and validated in an observational multicentre European hospital-based study. Elife. 2020;9.
- 24. Lippi G, Plebani M. The critical role of laboratory medicine during coronavirus disease 2019 (COVID-19) and other viral outbreaks. Clin Chem Lab Med. 2020;58(7):1063–9. pmid:32191623
- 25. Martens RJH, van Adrichem AJ, Mattheij NJA, Brouwer CG, van Twist DJL, Broerse J, et al. Hemocytometric characteristics of COVID-19 patients with and without cytokine storm syndrome on the sysmex XN-10 hematology analyzer. Clin Chem Lab Med. 2021;59(4):783–93. pmid:33554540
- 26. Wiersinga WJ, Rhodes A, Cheng AC, Peacock SJ, Prescott HC. Pathophysiology, Transmission, Diagnosis, and Treatment of Coronavirus Disease 2019 (COVID-19): A Review. JAMA. 2020;324(8):782–93. pmid:32648899
- 27. Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369:m1328. pmid:32265220
- 28. Zou H. The adaptive lasso and its oracle poperties. J Am Stat Assoc. 2006;101:1418–29.
- 29. Garcia-Finana M, Hughes DM, Cheyne CP, Burnside G, Stockbridge M, Fowler TA, et al. Performance of the Innova SARS-CoV-2 antigen rapid lateral flow test in the Liverpool asymptomatic testing pilot: population based cohort study. BMJ. 2021;374:n1637. pmid:34230058
- 30. Peto T, Team UC-LFO. COVID-19: Rapid antigen detection for SARS-CoV-2 by lateral flow assay: A national systematic evaluation of sensitivity and specificity for mass-testing. EClinicalMedicine. 2021;36:100924. pmid:34101770