Dear Editor,

During this unprecedented and ongoing pandemic, the need for healthcare professionals to prognosticate has arguably never been greater (1). Early risk-stratification of older adults admitted with COVID-19 is particularly important to mobilise healthcare professionals to manage their often complex care needs (2). Although frail older adults have higher mortality with COVID-19, given their inherent heterogeneity, predicting outcomes in this population is challenging (3). Multiple scales are available to risk-stratify patients with COVID-19 (4). Despite this, their predictive validity among those who are frail and hence, at highest risk of mortality and critical illness, is not yet established.

To examine this, we compared the accuracy of two recently validated risk-prediction instruments, the ISARIC-4C (5) and COVID-GRAM (6), in a sample of older adults admitted to three Irish hospitals (two acute and one rehabilitation). Consecutive patients aged ≥70 with laboratory-confirmed COVID-19 admitted between February 27th–April 24th 2020 were included. Frailty was determined by comprehensive geriatric assessment and stratified using the Clinical Frailty Scale (CFS). The ISARIC-4C is a new risk-prediction instrument incorporating eight variables scored on admission including age, sex at birth, respiratory rate, oxygen saturation, Glasgow Coma Scale score, blood urea, c-reactive protein value, obesity and number of co-morbidities based on the Charlson Co-morbidity Index (CCI). The COVID-GRAM includes 10 predictors including age, X-ray abnormalities, presence of haemoptysis, dyspnoea, level of consciousness, co-morbidities, history of cancer, neutrophil-lymphocyte ratio, serum lactate dehydrogenase level and direct serum bilirubin. The area under the receiver operating characteristic curve (AUROC) measured predictive accuracy for inpatient mortality, intensive care unit (ICU) admission and the composite outcome (critical illness). Logistic regression provided odds ratios (OR) per standard deviation (SD) increase with 95% confidence intervals (CI) exploring relationships between variables. Ethics approval was obtained (reference:ECM4(e) 05/05/2020).

In all, 69 patients were included, median age 70±10 years; 42% were female. The median CFS score was 5±2 and most (64%) were frail. Their demographic and clinical characteristics are presented elsewhere (7) but in summary, 15.9% (n=16%) had dementia, 58% (n=40) received polypharmacy (≥5 prescription medication) and most had high levels of co-morbidity; median CCI score 6±3. The majority (75%) had chest x-ray changes associated with COVID-19. In total, 16 (23%) died and 9 (13%) accessed ICU during their admission while 20 (29%) developed the composite outcome, critical illness. There was no significant difference in the proportion of frail and non-frail patients that died (22% versus 26%, p=0.77) or developed critical illness (24% versus 39%, p=0.26).

At initial diagnosis of COVID-19, 64% were scored as high-risk on the COVID-GRAM. The remainder as medium-risk. None were scored low-risk. Median COVID-GRAM scores were higher in those admitted to ICU (156 versus 210, p=0.01), who died (147 versus 198, p=0.002), or became critically ill (142 versus 198, p<0.001). Those scored at high-risk on the COVID-GRAM were significantly more likely to develop critical illness (p<0.001) or die during their admission (p=0.01). There was no difference in the proportion admitted to ICU (p=0.08). Results were similar for the ISARIC-4C; 8.5% were classed as intermediate-risk, 83% as high-risk and 8.5% as very high-risk, while no patients were scored low-risk. Median ISARIC-4C scores were significantly higher among those who died (10 versus 13, p<0.001) or developed critical illness (10 versus 13, p<0.001). There was no statistically significant difference between the COVID-GRAM (AUROC 0.76, 95% CI:0.65–0.88) and ISARIC-4C (AUROC 0.86, 95% CI:0.76–0.96) in predicting inpatient mortality (p=0.134). The ISARIC-4C was however, significantly more accurate than the COVID-GRAM in predicting those who developed critical illness (AUROC 0.90, 95% CI:0.82–0.97 versus 0.78, 95% CI:0.68–0.89, respectively, p=0.0486). ROC curves are presented in Figure 1. Diagnostic accuracy was similar when non-frail patients were excluded, although the difference in accuracy in predicting critical care was no longer significant (p=0.07). Adjusting for age, co-morbidity (CCI), sex, frailty, dementia and obesity, the COVID-GRAM (OR per SD increase 3.81, 95% CI:1.38–10.55, p=0.01) and ISARIC-4C (OR per SD increase 41.09, 95% CI:5.49–307.39, p<0.001), remained independent predictors of mortality. Similarly, after adjustment, both the COVID-GRAM (OR per SD increase 3.55, 95% CI: 1.39–9.07, p=0.008) and ISARIC-4C (OR 165.06, 95% CI: 9.94–2740.71, p<0.001), independently predicted those who would develop critical illness.

Figure 1
figure 1

Receiver operating characteristic curves comparing the predictive accuracy of the ISARIC-4C and COVID-GRAM for (a) death and (b) critical illness.

Full size image

In-keeping with the pattern of COVID-19 illness noted in other countries, frailty was common (8) and mortality high in this multi-morbid cohort of older inpatients. Those with higher COVID-GRAM and ISARIC-4C scores were more likely to die or develop critical illness during admission, even after adjustment for potential confounders. Although, both instruments had fair-good predictive validity, the ISARIC-4C had excellent and statistically greater accuracy in predicting critical illness among all but not frail patients, despite requiring fewer and arguably easier-to-obtain variables. Recent external validation of the COVID-GRAM also highlights that it overestimates risk in the highest-risk patients and has only fair accuracy for death (AUROC 0.79) (4) and critical illness (AUROC 0.72) (6), similar to our findings. However, clear ceiling effects were evident for both instruments such that no individuals were identified as low-risk, which may limit their utility in clinical practice. Although, this study is limited by the small sample size and risk of referral (admission rate) bias, it suggests that until these instruments are evaluated in large representative samples of older inpatients, they should only be used with caution to prognosticate and predict risk of critical illness in those who arguably are most likely to require it on admission, older patients. As Europe and other parts of the world face further waves of COVID-19 and until vaccination reduces risk, ongoing studies of prognostication models in frail older patients are needed.