Login Register Cart 0
Generic placeholder image

Coronaviruses

Editor-in-Chief

ISSN (Print): 2666-7967
ISSN (Online): 2666-7975

Back
Journal Home About Journal Editorial Board Current Issue Volumes/Issues
Subscribe
Systematic Review Article

A Critical Appraisal of the Risk of Bias in Systematic Reviews and Metaanalyses Pertaining to COVID-19

Author(s): Amit Dang*, Sheshank Madiraju, Jagan Mohan Venkateswara Rao P, Navya Sri Gurram, Sandeep Digijarala, Sumit Dang and Vallish B.N.

Volume 3, Issue 2, 2022

Published on: 30 December, 2020

Article ID: e301220189630 Pages: 7

DOI: 10.2174/2666796701666201230105144

Abstract

Background: We critically evaluated the risk of bias in published systematic reviews (SRs) and meta-analyses (MAs) pertaining to COVID-19 using ROBIS tool.

Materials And Methods: MEDLINE and Cochrane Central Library were searched for SRs/MAs on 14th May 2020, including studies of all designs describing various facets of COVID-19 in humans; no restrictions were applied for interventions, comparators, and outcomes. Two reviewers independently assessed all the SRs/MAs with ROBIS.

Results: Out of 204 identified records, 48 SRs/MAs were included. The most frequently reviewed topics were therapy outcomes, diagnosis, and comorbidities (15, 8, and 6 papers respectively). Only 29/48(60.41%) papers had made a mention of using PRISMA or other guidelines for drafting the SR/MA. Only 5/48(10.42%) of all included SRs/MAs had low overall risk of bias as per ROBIS tool; 41/48(85.42%) had high risk of bias, 2/48(4.17%) had unclear risk of bias. The highest proportion of bias was found in data synthesis and findings (30/48, 62.50% of studies had high risk of bias), followed by study identification and selection (29/48, 60.42%). The IRR for methodological quality assessment was substantial, with the Cohen’s kappa values being 0.64, 0.68, 0.62, and 0.75 for domains 1-4 of ROBIS tool, and 0.66 for overall risk of bias assessment.

Conclusion: There are serious concerns about the methodology employed to generate the results of the SRs/ MAs pertaining to COVID-19, with ‘quantity’ seemingly being given more importance than ‘quality’ of the paper.

Keywords: COVID-19, systematic reviews, meta-analysis, ROBIS, comorbidities, methodological quality.

Graphical Abstract

[1]
Hasan H, Muhammed T, Yu J, et al. Assessing the methodological quality of systematic reviews in radiation oncology: A systematic review. Cancer Epidemiol 2017; 50(A): 141-9.
[http://dx.doi.org/10.1016/j.canep.2017.08.013]
[2]
Kang H. Statistical considerations in meta-analysis. Hanyang Med Rev 2015; 35: 23-32.
[http://dx.doi.org/10.7599/hmr.2015.35.1.23]
[3]
Lander B, Balka E. Exploring how evidence is used in care through an organizational ethnography of two teaching hospitals. J Med Internet Res 2019; 21(3): e10769.
[http://dx.doi.org/10.2196/10769] [PMID: 30920371]
[4]
Braga LH, Pemberton J, Demaria J, Lorenzo AJ. Methodological concerns and quality appraisal of contemporary systematic reviews and meta-analyses in pediatric urology. J Urol 2011; 186(1): 266-71.
[http://dx.doi.org/10.1016/j.juro.2011.03.044] [PMID: 21600615]
[5]
Gagnier JJ, Kellam PJ. Reporting and methodological quality of systematic reviews in the orthopaedic literature. J Bone Joint Surg Am 2013; 95(11): e771-7.
[http://dx.doi.org/10.2106/JBJS.L.00597] [PMID: 23780547]
[6]
Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009; 339: b2535.
[http://dx.doi.org/10.1136/bmj.b2535] [PMID: 19622551]
[7]
Whiting P, Savović J, Higgins JPT, et al. ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016; 69: 225-34.
[http://dx.doi.org/10.1016/j.jclinepi.2015.06.005] [PMID: 26092286]
[8]
Bühn S, Mathes T, Prengel P, et al. The risk of bias in systematic reviews tool showed fair reliability and good construct validity. J Clin Epidemiol 2017; 91: 121-8.
[http://dx.doi.org/10.1016/j.jclinepi.2017.06.019] [PMID: 28694122]
[9]
Mangolini VI, Andrade LH, Lotufo-Neto F, Wang YP. Treatment of anxiety disorders in clinical practice: a critical overview of recent systematic evidence. Clinics (São Paulo) 2019; 74: e1316.
[http://dx.doi.org/10.6061/clinics/2019/e1316] [PMID: 31721908]
[10]
Sacristán JA, Dilla T, Díaz-Cerezo S, Gabás-Rivera C, Aceituno S, Lizán L. Patient-physician discrepancy in the perception of immune-mediated inflammatory diseases: rheumatoid arthritis, psoriatic arthritis and psoriasis. A qualitative systematic review of the literature. PLoS One 2020; 15(6): e0234705.
[http://dx.doi.org/10.1371/journal.pone.0234705] [PMID: 32555708]
[11]
Li J, Li YX, Luo LJ, et al. The effectiveness and safety of acupuncture for knee osteoarthritis: An overview of systematic reviews. Medicine (Baltimore) 2019; 98(28): e16301.
[http://dx.doi.org/10.1097/MD.0000000000016301] [PMID: 31305415]
[12]
Alhersh E, Abushanab D, Al-Shaibi S, Al-Badriyeh D. Caffeine for the treatment of apnea in the neonatal intensive care unit: a systematic overview of meta-analyses. Paediatr Drugs 2020; 22(4): 399-408. [published online ahead of print, 2020 Jun 3].
[http://dx.doi.org/10.1007/s40272-020-00404-4] [PMID: 32488731]
[13]
Uman LS. Systematic reviews and meta-analyses. J Can Acad Child Adolesc Psychiatry 2011; 20(1): 57-9.
[PMID: 21286370]
[14]
Ranganathan P, Pramesh CS, Aggarwal R. Common pitfalls in statistical analysis: Measures of agreement. Perspect Clin Res 2017; 8(4): 187-91.
[http://dx.doi.org/10.4103/picr.PICR_123_17] [PMID: 29109937]
[15]
Misra DP, Agarwal V. Systematic reviews: challenges for their justification, related comprehensive searches, and implications. J Korean Med Sci 2018; 33(12): e92.
[http://dx.doi.org/10.3346/jkms.2018.33.e92] [PMID: 29542301]
[16]
Han JL, Gandhi S, Bockoven CG, Narayan VM, Dahm P. The landscape of systematic reviews in urology (1998 to 2015): an assessment of methodological quality. BJU Int 2017; 119(4): 638-49.
[http://dx.doi.org/10.1111/bju.13653] [PMID: 27611722]
[17]
Remschmidt C, Wichmann O, Harder T. Methodological quality of systematic reviews on influenza vaccination. Vaccine 2014; 32(15): 1678-84.
[http://dx.doi.org/10.1016/j.vaccine.2014.01.060] [PMID: 24513008]
[18]
Samargandi OA, Hasan H, Thoma A. Methodologic quality of systematic reviews published in the plastic and reconstructive surgery literature: a systematic review. Plast Reconstr Surg 2016; 137(1): 225e-36e.
[http://dx.doi.org/10.1097/PRS.0000000000001898] [PMID: 26710056]
[19]
Wasiak J, Shen AY, Ware R, O’Donohoe TJ, Faggion CM Jr. Methodological quality and reporting of systematic reviews in hand and wrist pathology. J Hand Surg Eur Vol 2017; 42(8): 852-6.
[http://dx.doi.org/10.1177/1753193417712660] [PMID: 28610464]
[20]
Schroter S, Price A, Flemyng E, et al. Perspectives on involvement in the peer-review process: surveys of patient and public reviewers at two journals. BMJ Open 2018; 8(9): e023357.
[http://dx.doi.org/10.1136/bmjopen-2018-023357] [PMID: 30185581]
[21]
Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med 2006; 99(4): 178-82.
[http://dx.doi.org/10.1177/014107680609900414] [PMID: 16574968]
[22]
Chung KJ. Preprints: What is their role in medical journals? Arch Plast Surg 2020; 47(2): 115-7.
[http://dx.doi.org/10.5999/aps.2020.00262] [PMID: 32203987]
[23]
Fry NK, Marshall H, Mellins-Cohen T. In praise of preprints. Microb Genom 2019; 5(4): e000259.
[http://dx.doi.org/10.1099/mgen.0.000259] [PMID: 30938670]

Rights & Permissions Print Cite
Article Metrics
20
1
© 2024 Bentham Science Publishers | Privacy Policy