Development and Validation of a Nomogram to Assist Monitoring Nosocomial SARS-CoV-2 Infection of Hospitalized Patients

Introduction

In December 2019, the outbreak of novel coronavirus disease 2019 (COVID-19) occurred in Wuhan, China, which is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Since then, it began to spread rapidly in Hubei Province, throughout China and abroad areas, which has turned into a worldwide public health emergency and reached pandemic status.¹ By the time of November 22, 2021, the number of reported cases has surpassed 256, 480, 022 with over 5, 145, 002 deaths worldwide.²

SARS-CoV-2 is extremely infectious and spread primarily through respiratory droplets and close exposure. As patients possibly infected with SARS-CoV-2 need to attend hospital, the incidence of nosocomial infection is conceivably high.³ Most patients infected with SARS-CoV-2 in hospitals were healthcare workers, among them nurses constituted the largest group, followed by doctors. Intensive surveillance of nosocomial SARS-CoV-2 infection in hospitalized patients is imperative to provide appropriate supportive care for patients, as well as prevent virus further transmission and avoid cross-contamination with non-infected individuals. So far, viral RNA detection by real-time quantitative PCR (RT-qPCR) and serological antibody test are most popular lab tests to confirm the diagnosis of SARS-CoV-2 infection.⁴ The positive detection rate of nucleic acid is only about 30%–50%, mainly owing to many factors, such as sample types, different disease phages and techniques for specimens collecting and testing.^5–7 Therefore, tests need to be repeated many times in suspected patients with initial negative results.⁸ In addition, the procedures for detection are laborious and time-consuming. In contrast with RT-qPCR, serological antibodies test is beneficial with high reproducibility, short turn-around time and high biosecurity owing to the homogeneity of blood samples.^9,10 Nevertheless, IgM and IgG antibodies titers are hinged largely on the time of antibody responses after infection, and they can be negative during the window periods.¹¹ Despite recommendations to test hematological and biochemical indicators as part of the diagnostic workup for COVID-19,^12,13 whether the routine laboratory tests could correctly distinguish cases of COVID-19 from non-infected inpatients is still doubtful. Our laboratory previously analyzed the hematological and biochemical parameters of 84 COVID-19 patients and 221 patients with community-acquired pneumonia (CAP), which indicated that most blood indexes of COVID-19 patients displayed obvious differences in contrast with CAP patients or healthy controls and some parameters could be predictive of COVID-19 diagnosis.¹⁴ Nevertheless, the individual indicator only had a fair discriminatory ability. To further evaluate the diagnostic value of the blood routine test, we constructed and validated a nomogram integrated eleven laboratory indexes to assist monitoring nosocomial SARS-CoV-2 infection in hospitalized patients.

Patients and Methods

Data Collection

Clinical data and laboratory indicators of 437 COVID-19 hospitalized cases and 420 negative inpatients between January 20th 2020 and May 25th 2020 were retrospectively collected from two clinical Centers: Zhongnan Hospital of Wuhan University and Taihe Hospital of Hubei University of Medicine. Clinical laboratory indicators of routine blood tests and blood biochemical parameters for the first time after admission were collected from routine clinical practice. All recruited COVID-19 patients were diagnosed based on the Guidelines for Diagnosis and Treatment of Novel Coronavirus Pneumonia (7th version). All control groups were negative inpatients for SARS-CoV-2 RNA and antibodies, and patients with hematologic diseases or radio-chemotherapy or critical illness were excluded. The study complied with the Declaration of Helsinki and was approved by the Ethics Committees of Zhongnan Hospital of Wuhan University (No. ZN2020015) and the Medical Ethics Committee of Taihe Hospital of Hubei University of Medicine (No. TH2020009). Informed written consent was waived due to the use of de-identified retrospective data.

Laboratory Methods

Clinical laboratory indicators consisted of white blood cell count (WBC), red blood cell count (RBC), hemoglobin (HGB), platelet (PLT), neutrophil count (NEUT), lymphocyte count (LYMPH), monocyte count (MONO), eosinophil count (EO), basophil count (BASO), hematocrit (HCT), mean corpuscular volume (MCV), mean corpuscular hemoglobin (MCH), mean corpuscular hemoglobin concentration (MCHC), red blood cell distribution width coefficient variation (RDW-CV), mean platelet volume (MPV), albumin (ALB), globulin (GLB), alanine transaminase (ALT), aspartate aminotransferase (AST), blood urea nitrogen (BUN), creatinine (Cr), C-reactive protein (CRP), total bilirubin (TBIL), direct bilirubin (DBIL), and total bile acid (TBA). All hematology and biochemical indicators were analyzed by standard automatic analyzer using commercial kits according to the manufacturers’ protocols. SARS-CoV-2 RNAs were extracted from throat swabs and tested using reverse transcription real-time quantitative PCR assays (RT-qPCR). SARS-CoV-2 antibodies were detected by colloidal gold kits or chemiluminescence techniques according to the manufacturer’s instruction.

Statistical Analysis

Categorical variables were shown to be frequencies with proportions, and Chi-square test or Fisher’s exact test were conducted to analyze the differences, as appropriate. Continuous variables were tested for normality distribution by the Kolmogorov–Smirnov test and were shown as mean ± standard deviation (SD) or median with interquartile range (IQR) based on distributions. Independent t-test and Mann–Whitney U-test were tested for continuous variables with or without normal distributions, respectively. Two-sided P < 0.05 was considered to be statistically significant. All the data analysis was performed using R software (version 3.6.0).

Least absolute shrinkage and selection operator (LASSO) regression analysis was applied to identify relatively important features. The prediction model was constructed by logistic regression, support vector machine (SVM), decision tree classification and random forest (RF). The nomogram was built using rm packages, and the discriminative ability was assessed by the area under the receiver operating characteristic curve (ROC).¹⁵ The nomogram was validated internally in the development cohort and externally in the validation cohort. The calibration curves were utilized to compare the correlation between the predicted probabilities and the actual outcomes.¹⁶ Decision curve analysis (DCA) and clinical impact curve analysis (CICA) were implemented to assess clinical usability and benefits of the nomogram.^17,18

Results

Demographics and Clinical Characteristics of Participants in the Development and Validation Cohorts

In the development cohort, 296 COVID-19 hospitalized cases and 305 negative inpatients were enrolled from Zhongnan Hospital of Wuhan University. A validation cohort consisting of 141 positive COVID-19 hospitalized cases and 115 negative inpatients was recruited from Taihe Hospital of Hubei University of Medicine. In two cohorts of our study, the distributions of age, gender and disease types were similar between cases and controls. Among the control group, both cohorts consisted of about 20% patients with community-acquired pneumonia, including bacterial pneumonia, fungal pneumonia, pneumocystosis or chlamydia pneumonia. In the development cohort, COVID-19 patients had significantly higher value for RBC, HGB, HCT, MCH, MCHC, ALT, AST, Cr and lower value for WBC, PLT, NEUT, LYMPH, MONO, EO, BASO, RDW, MPV, GLB and BUN than controls. There were no significant differences in MCV, ALB, CRP, TBIL, DBIL and TBA between cases and controls (Table 1). In the validation cohort, COVID-19 patients show increased RBC, HGB, HCT, ALB, Cr and CRP, and decreased WBC, NEUT, EO, BASO, RDW, MPV, BUN, TBIL and TBA, compared with controls. The level of PLT, LYMPH, MONO, MCV, MCH, MCHC, GLB, ALT, AST and DBIL were comparable between COVID-19 patients and controls (Table 1).

Table 1 Demographics and Clinical Characteristics of Participants in the Development and Validation Cohorts

Features Selection and Machine Learning Model Evaluation

Thirty variables measured at hospital admission were collected from each participant in the development cohort. After rejecting unrelated and superfluous variables, twenty variables were included in the LASSO regression analysis (Figure 1A). Using the LASSO regression filtration, eleven variables remain significant predictors of positive cases, including RBC, HGB, PLT, NEUT, LYMPH, MONO, RDW, MPV, ALB, GLB and CRP (Figure 1B). Then, we constructed prediction models using logistic regression, support vector machine (SVM), decision tree classification, random forest (RF), and the precision–recall curve (Figure 2) and the receiver operating characteristic curve (ROC) (Figure 3) were used to assess their accuracy. There were no obvious differences in the accuracy of these four models, and we chose the logistic regression model for further analysis due to its high interpretability.

Figure 1 Construction of predictive nomogram in hospitalized patients with SARS-CoV-2 infection. (A) Least absolute shrinkage and selection operator (LASSO) coefficients profiles (y-axis) of the twenty high dimensional predictors, which uses a L1 penalty to shrink some regression coefficients to exactly zero. The upper x-axis is the average numbers of predictors and the lower x-axis is the log(λ). (B) Fivefold cross-validation for tuning parameter selection in the LASSO model. The area under the receiver operating characteristic (AUC) with error bar is plotted against log(λ), where λ is the tuning parameter. The dotted vertical lines are drawn at the optimal values by minimum criteria and the one standard error of the minimum criteria (1se criteria). The upper x-axis is the average numbers of predictors and the lower x-axis is the log(λ). To avoid overfitting, 1se criteria (λ=0.031) was selected. (C) Nomogram predicted SARS-CoV-2 infection of hospitalized patients with eleven laboratory indicators, including RBC, HGB, PLT, NEUT, LYMPH, MONO, RDW, MPV, ALB, GLB and CRP. Each level of every variable was conferred a score on the scale of points. By summing up the scores of selected variables, a total score was available. The prediction possibility corresponding to this total score was used to predict SARS-CoV-2 infection of hospitalized patients. Abbreviations: RBC, red blood cell count; HGB, hemoglobin; PLT, platelet; NEUT, neutrophil count; LYMPH, lymphocyte count; MONO, monocyte count; RDW, red blood cell distribution width coefficient variation; MPV, mean platelet volume; ALB, albumin; GLB, globulin; CRP, C-reactive protein.

Figure 2 The precision-recall curve of four machine learning model. (A) Logistic regression. (B) Support vector machine (SVM). (C) Decision tree classification. (D) Random forest (RF).

Figure 3 The ROC curve of four machine learning model. (A) Logistic regression. (B) Support vector machine (SVM). (C) Decision tree classification. (D) Random forest (RF).

Construction, Evaluation and External Validation of a Nomogram

The predictive nomogram based on the selected eleven variables from the development cohort was built to predict SARS-Cov-2 infection (Figure 1C). Each level of every variable was conferred a score on the scale of points. By summing up the scores of selected variables, a total score was available. The prediction possibility corresponding to this total score was used to predict SARS-CoV-2 infection in hospitalized patients. The calibration plot of the nomogram displayed that the predicted outcomes were in good concordance with the actual observations both in the development cohort and validation cohort (Figure 4A and B). The results of DCA and CICA indicated that the nomogram showed a better net benefit across a wide range of threshold probabilities and affected the outcome of patients (Figure 4C and D).

Figure 4 The calibration, DCA and CICA curves of the nomogram. (A) The calibration curve of the nomogram in the development cohort. (B) The calibration curve of the nomogram in the validation cohort. (C) The decision curve analysis (DCA) of the nomogram in the development cohort. The y-axis represents the net benefit and the x-axis represents the predicted probability of SARS-CoV-2 infection: a perfect prediction model (grey line), screen none (black line) and screen based on the nomogram (red line) (D) The clinical impact curve analysis (CICA) of the nomogram in the development cohort. The red line (number of high-risk individuals) is the number of people who are classified as positive by the model at each threshold probability, the blue line (number of high-risk individuals with event) indicates the number of true positives at each threshold probability.

In the development cohort, the nomogram showed a high AUC 0.863 (95% CI: 0.834–0.892) to identify COVID-19 patients from hospitalized patients, with a sensitivity of 80.41% and specificity of 77.38% (Figure 5A). In the validation cohort, AUC was 0.813 (95% CI: 0.760–0.866) for SARS-CoV-2 positive patients with a sensitivity of 82.98% and specificity of 70.43% (Figure 5B).

Figure 5 The ROC curve of the nomogram. (A) The ROC curve of the nomogram in the development cohort. (B) The ROC curve of the nomogram in the validation cohort.

Discussion

Intensive surveillance of nosocomial SARS-CoV-2 infection in hospitalized patients is of great significance and may assist in providing patients with appropriate supportive care and preventing further virus transmission. In the current study, we identified eleven laboratory indicators, including RBC, HGB, PLT, NEUT, LYMPH, MONO, RDW, MPV, ALB, GLB and CRP, were associated with SARS-CoV-2 infection. Furthermore, we developed and validated an effectual nomogram integrated eleven features to predict SARS-CoV-2 infection in hospitalized patients with high sensitivity and specificity. The calibration plot showed a good consistency between the actual observations and the predicted outcomes both in the development cohort and validation cohort. DCA and CICA further implied that the nomogram displayed a better net benefit across a wide range of threshold probabilities and impacted the outcome of patients.

The standard approach for the etiological diagnosis of SARS-CoV-2 infection is nucleic acid detection by RT-qPCR. Although some patients were strongly epidemiologically correlated with SARS-CoV-2 exposure and presented symptoms of suspected viral pneumonia, the nucleic acid test of respiratory tract specimens did not show positive results generally until 5–6 days of symptom onset. Additionally, the sensitivity of nucleic acid test was also determined by different disease phages, sample types and techniques for collection, storage, transport and detection.^5–7 Therefore, tests need to be repeated several times in patients with suspicion of being infected before confirming diagnosis, and they might not have enough priority to receive appropriate supportive care and isolation managements.¹⁹ In addition, the procedures of detection are laborious, and results may take at least several hours to be obtained. In contrast with RT-qPCR, Immunological detection of IgM/IgG against SARS-CoV-2 is beneficial with short turn-around time, high biosecurity and high reproducibility owing to the homogeneity of blood samples.^9,10 However, IgM and IgG antibody titers are hinged largely on the time of antibody responses after infection, and they can be negative during the window periods.¹¹ Furthermore, serological antibody test cannot confirm SARS-CoV-2 existence and only provides evidence of recent infection.²⁰ Our nomogram based on daily blood routine laboratory tests may be helpful to monitor nosocomial SARS-CoV-2 infection and adopt timely isolation measures, while confirmatory results are not available.

Our nomogram integrated eleven easy-access laboratory indicators at hospital admission, including higher RBC, HGB, CRP and lower PLT, NEUT, LYMPH, MONO, RDW, MPV, ALB and GLB, similar results were obtained from the early investigation of the blood indexes of 84 COVID-19 patients and 221 CAP patients conducted by our laboratory, which signified that most blood parameters of COVID-19 patients showed noticeable discrepancies in contrast with CAP patients or healthy controls and some indexes could be predictive of COVID-19 diagnosis.¹⁴ Previous studies have already found that platelets, lymphocytes, neutrophil, monocyte, red blood cell distribution width coefficient variation, albumin and C-reactive protein are changeable at different degrees in COVID-19 patients.^21–24 Hypoalbuminemia, lymphopenia and CRP ≥ 4 mg/dL were reported as the predictive variables for pneumonia development against respiratory failure in patients with MERS-CoV infection.²⁵ Liu et al indicated that lymphocytes count and albumin were negatively associated with the Murray scores, which were originally developed to evaluate the severity of acute lung injury, while C reactive protein level was positive with the Murray scores in COVID-19 patients.²⁶ SARS-CoV-2 might work on lymphocytes, as does SARS-CoV, which elicits a cytokine storm and contributes to a series of immune responses.²⁷ The decline of T lymphocyte count in COVID-19 patients implied that SARS-CoV-2 depleted immune cells and restrained the cellular immune function.^22,28 In our previous study, we found that elevated RDW was a valuable prognostic marker for severe COVID-19.²⁴ However, this study showed a decreased RDW in early SARS-CoV-2 infected patients comparing with non-COVID-19 patients. Therefore, further detailed study should be conducted to elucidate how the SARS-CoV-2 induces the dynamic change of blood components and which blood tests are more suggestive for differential diagnosis.

Our study has some advantages. First, with daily blood routine tests available, we could track and monitor inpatients that may have developed COVID-19, allowing to take more effective isolation measures to prevent virus further transmission. Second, to ensure the reliability of the conclusion, our nomogram was constructed with a relatively large sample size and validated externally using other populations. The performance of our nomogram was effectual for clinical application.

Several potential limitations of our study should be acknowledged. First, this is a retrospective study, which may have some inherent biases. Second, it was estimated that about 60% of SARS-CoV-2 infections are asymptomatic cases, possibly transmitting the virus before displaying any symptoms.^29,30 However, clinical data and laboratory indicators of our study were collected during hospital admission, with patients having already presented symptoms and signs of COVID-19. Due to the lack of data on asymptomatic patients, we doubted whether our nomogram is suitable for asymptomatic infected individuals. Finally, the cohorts for development and validation were entirely from Hubei, China, which could restrict the generalizability of the nomogram in other areas of China. Therefore, further validation studies from domestic and abroad areas should be completed.

Conclusion

In summary, we constructed and validated a nomogram that integrated eleven laboratory indexes to assist monitoring SARS-CoV-2 infection of hospitalized patients. Our nomogram is extremely informative for clinical practice, which will be helpful for preventing SARS-CoV-2 further transmission in hospital and avoiding nosocomial infection.