ABSTRACT

BACKGROUND:

Numerous systematic reviews on coronavirus disease-19 (COVID-19) treatment have been developed to provide syntheses of the large volume of primary studies. However, the methodological quality of most of these reviews is questionable and the results provided may therefore present bias.

OBJECTIVE:

To investigate how many systematic reviews on the therapeutic or preventive options for COVID-19 assessed the certainty of the evidence through the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.

METHODS:

We conducted a sensitive search in MEDLINE (via PubMed) and included all systematic reviews that assessed any intervention for COVID-19. The systematic reviews included were examined to identify any planned and/or actual assessment using the GRADE approach (or absence thereof) regarding the certainty of the evidence.

RESULTS:

We included 177 systematic reviews and found that only 37 (21%; 37/177) assessed and reported the certainty of the evidence using the GRADE approach. This number reduced to 27 (16.2%; 27/167) when Cochrane reviews (n = 10), in which an evaluation using GRADE is mandatory, were excluded.

CONCLUSION:

Most of the systematic reviews on interventions relating to COVID-19 omitted assessment of the certainty of the evidence. This is a critical methodological omission that must not be overlooked in further research, so as to improve the impact and usefulness of syntheses relating to COVID-19.

KEY WORDS (MeSH terms):
COVID-19; Systematic review [publication type]; Evidence-based practice; GRADE approach; Evidence-based medicine; Public health

AUTHORS’ KEY WORDS:
Methodology; Certainty of evidence; Strength of recommendations; Evidence-based healthcare

INTRODUCTION

Since the beginning of the coronavirus disease 2019 (COVID-19) pandemic, large numbers of studies have been published in an attempt to find an effective treatment for this disease. Consequently, many systematic reviews have been developed on this topic, to provide syntheses of the large volume of primary studies. Healthcare professionals and policymakers commonly use systematic reviews to formulate recommendations and make practical decisions.¹1. Schünemann HJ, Santesso N, Vist GE, et al. Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what. J Clin Epidemiol. 2020:127:202-7. PMID: 32512187; http://doi.org/10.1016/j.jclinepi.2020.05.030.
http://doi.org/10.1016/j.jclinepi.2020.0...

However, the methodological quality of most of these systematic reviews is questionable. Hence, the results provided through these reviews may present bias.

Assessing the certainty of the evidence is an indispensable step in a systematic review. This is especially true within the current context, in which information is often misleading yet has been widely disseminated, both by scientific journals and by the traditional media. Thus, efforts need to be made by the authors of syntheses of the evidence on a given topic, to ensure that the degree of certainty that can be placed on the estimates of effect and clinical recommendations can be established.¹1. Schünemann HJ, Santesso N, Vist GE, et al. Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what. J Clin Epidemiol. 2020:127:202-7. PMID: 32512187; http://doi.org/10.1016/j.jclinepi.2020.05.030.
http://doi.org/10.1016/j.jclinepi.2020.0... ^,22. Kumar A, Taggarsi M. GRADEing quality of evidence and its importance in evidence-based practice. BMJ Evid Based Med. 2020:bmjebm-2020-111525. PMID: 33004427; http://doi.org/10.1136/bmjebm-2020-111525.
http://doi.org/10.1136/bmjebm-2020-11152...

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) is a transparent approach to rating the certainty of the body of evidence in systematic reviews and other forms of synthesis, as a guide to making decisions.³3. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383-94. PMID: 21195583; http://doi.org/10.1016/j.jclinepi.2010.04.026.
http://doi.org/10.1016/j.jclinepi.2010.0... This approach should be beneficial during the COVID-19 pandemic. It is entirely possible to perform GRADE assessments, even (or even more so) within an emergency context.¹1. Schünemann HJ, Santesso N, Vist GE, et al. Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what. J Clin Epidemiol. 2020:127:202-7. PMID: 32512187; http://doi.org/10.1016/j.jclinepi.2020.05.030.
http://doi.org/10.1016/j.jclinepi.2020.0... ^,44. Munn Z, Twaddle S, Service D, et al. Developing guidelines before, during, and after the COVID-19 pandemic. Ann Intern Med. 2020:173(12):1012-4. PMID: 32931327; http://doi.org/10.7326/M20-4907.
http://doi.org/10.7326/M20-4907...

OBJECTIVE

We carried out a critical appraisal study with the aim of investigating how many systematic reviews that have been published in relation to therapeutic or preventive options for COVID-19 made assessments of the certainty of the evidence through the GRADE approach.

METHODS

We conducted a sensitive search in MEDLINE (via PubMed) on January 20, 2021, using the MeSH term “Coronavirus” and its synonyms combined with the PubMed clinical queries filter for systematic reviews (Annex 1). Two authors independently screened all titles and abstracts through the Rayyan platform,⁵5. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. PMID: 27919275; http://doi.org/10.1186/s13643-016-0384-4.
http://doi.org/10.1186/s13643-016-0384-4... in order to include systematic reviews that assessed any intervention for COVID-19.

RESULTS

The systematic reviews included were analyzed in full text, to identify whether there was any planned and/or actual assessment of the certainty of the evidence using the GRADE approach (or absence thereof). The search strategy found 1,075 references, and 177 fulfilled the inclusion criteria. Of these, only 37 reviews (21%; 37/177) assessed the certainty of the evidence using the GRADE approach. This number reduced to 27 (16.2%; 27/167) when Cochrane reviews (n = 10), in which an evaluation using GRADE is mandatory, were excluded.

CONCLUSION

This result highlights the fact that most of the systematic reviews on interventions conducted in relation to COVID-19 omitted assessment of the certainty of the evidence. This is a critical methodological omission that must not be overlooked in further research, so as to improve the impact and usefulness of syntheses relating to COVID-19.

Annex 1

REFERENCES

Publication Dates

History