commentary

How Riding the Wave of COVID-19 Pandemic has Affected the Quality of Evidence-Based Medicine in Surgical Journal Publications

Authors: Michele Altomare1,2*, Stefano Piero Bernardo Cioffi2, Shir Sara Bekhor2, Ludovica Bartiromo3, Andrea Spota2, Andrea Piero Chierici4, Matteo Schimberni3

*Corresponding Author: Michele Altomare, General Surgery and Trauma Team, ASST Niguarda, Milano, Piazza Ospedale Maggiore 3, 20162 Milan, Italy

Received Date: 25 February, 2022

Accepted Date: 02 March, 2022

Published Date: 07 March 2022

Citation: Altomare M, Cioffi SPB, Bekhor SS, Bartiromo L, Spota A, et al. (2022) How Riding the Wave of COVID-19 Pandemic has Affected the Quality of Evidence-Based Medicine in Surgical Journal Publications. Ann Case Report 7: 791. DOI: https://doi.org/10.29011/2574-7754.100791

Abstract

COVID-19 era is one of the most disturbing in recent memory. The scientific community has re-acted quickly. During this critical situation, COVID-19 highlighted criticisms affecting Evidence-Based Medicine in our time clearer than in the past. The present short communication brings to light some of the issues related to the current citation indexing-based system of publication, reporting critical bias that affected the first and second waves of the pandemic.

Keywords: COVID-19 pandemic; Publishing; Surgery; Obstetrics and gynecology; Citation index; H-index; Evidence-Based Medicine

Introduction

The COVID-19 pandemic has profoundly changed many aspects of our lives over the past two years. Each era comes with its challenges, but the ongoing COVID-19 pandemic stands out as one of the most disturbing in recent memory. Since the beginning of the out-break, the scientific community has invested tremendous efforts and resources to foster our understanding of the virus and develop a vaccine. These outstanding efforts catalysed the production of several anti-COVID vaccines in record time. While some argue that speeding up randomized clinical trials was an unavoidable result of the urgency of the situation, some say that the experience from the pandemic has created methods that enhance the efficiency in drug and vaccine development, which are here to stay. The pandemic seems to have affected scientific research by the medical community (and not only) in more than one way, leading to the production of 160,000 COVID-related articles only in 2020. Looking back at March 2020, when the WHO declared the outbreak a pandemic and Italy introduced its first nationwide lockdown, our daily practice was quickly altered, and we faced an unknown situation [1]. As new and aspiring surgeons in general/emergency surgery and obstetrics/gynecology (OB/GYN) in two major hospitals in Milan (Lombardy, Italy), our clinical activities and surgical learning curve were significantly affected during the first wave in particular [2,3].

When traditional surgical experiences were replaced by assisting with the ongoing crisis, we consulted the most recent literature to be kept up to date. However, we found that medical journals were saturated with pandemic-related articles and that a large portion of these papers did not appear to have a concrete clinical application. This immediately brought to mind the historical criticism of Evidence-Based Medicine (EBM). Although in times of emergency there is a need for quick dissemination of novel findings, we fear that authors have spotted an opportunity for publishing apparently COVID-related papers and to “ride the wave”. For this reason, and in order to understand the value of articles published during this critical period in our fields of practice, we decided to analyse available literature and select papers based on their clinical relevance. An advanced systematic search was performed in accordance with PRISMA guidelines for systematic reviews and meta-analyses [4]. Using the PubMed search query “[((COVID-19 OR COVID OR SARS-CoV-2) AND (surgery or gynecology or infective disease or anesthesiology or pneumology))],” we found that in the period from February to April 2020 (first wave in Italy), alone, more than 6,000 articles were published. We immediately realized that the electronic database was overwhelmed with papers in many different fields, mostly written by specialists who usually do not play any role in the management of infectious diseases or respiratory syndromes. Almost one-quarter (1,485) of these articles were related to surgical fields. After the abstract and full-text screening, 63 articles focusing on the management of surgical COVID-19 patients were identified. The Oxford Centre for Evidence-Based Medicine (OCEBM) levels of evidence were utilized to select the most reliable works [5], and 54/63 (85%) of the articles were graded as level 5, the weakest level of evidence, even though the search was restricted to journals with an h-index over the mean according to Scimago Journal & Country Ranking (surgery: mean h-index, 25; OB/GYN: mean h-index, 27).

Therefore, due to the very low level of evidence of the available results, we decided not to proceed with a systematic review or a meta-analysis. In addition, we found that none of the 11 articles claiming to establish guidelines/recommendations actually met the purpose of guidelines, that is, to provide evidence-based recommendations to guide clinical choices. In reality, most of these papers are expert opinions without any established scientific evidence underlying the drafted recommendations. No Grades of Recommendation Assessment, Development, and Evaluation (GRADE) or other reliability scores were re-ported. Moreover, all the articles selected were published in high-quality journals with an H-Index consistently over the mean for surgical and Ob-Gyn journals. The preliminary results of our research are summarized in Appendix A and Appendix B. EBM relies on the principle that what is justifiable or reasonable to believe depends on the trustworthiness of the evidence [6]. Today, many tools are available to ensure that articles are evidence-based, including checklists and bias control statements, which are used to minimize the risks of error and submission of irrelevant/inaccurate papers. The ongoing pandemic must not undermine this basic principle or be used as an excuse to publish for the sake of publishing. Unfortunately, concepts such as citation indexing have severely affected some authors’ perceptions and tendency toward publishing innovative findings, shifting the dogma to “citation-catching” contributions. The best publications are not necessarily the most cited.

Furthermore, attempting to take advantage of the momentum to raise one’s personal bibliometric index leads to the production of meaningless work. Scientific articles driven by these motives risk lowering the quality of available literature and burying significant papers in an ocean of insignificant ones. During the first wave of the COVID-19 pandemic, the median time-to-acceptance of the articles selected in our search was four days. Journals were overwhelmed by an excess of submissions while attempting to balance undertaking a valid and thorough peer review with rapid publication of relevant news about the pandemic. These contrasting aims contributed to the publication of many papers that do not appear to provide practical clinical benefits. In the last year, global collaborative groups were built to offer high-quality standards in surgical contributions related to the pandemic [7,8]. Hopefully, in the near future, this willingness to cooperate, which we have learned from a global emergency, will be used to design multicentric works in all clinical and translational surgical research fields. 

In conclusion, although the scientific community has provided major breakthroughs in this challenging period, the quality of newly published guidelines and recommendations should not be compromised. Especially in the context of evolving fields such as emergency medicine, scientific literature must maintain a high level of evidence and clinical relevance while avoiding unreliable work driven by an author’s wish for increased visibility or h-index. It would be interesting to compare the number of submitted articles and the trustworthiness of published ones if scientific journals decided to block the citation on the articles with “COVID-19” in the title. During emergencies, other sources and media channels should be considered platforms to preserve the reliability of scientific journals [9]. Surgical scientific societies should take all necessary efforts to consider personal experiences and expert opinions in their specific context and reduce the dissemination of irrelevant publications.

Author Contributions

Conceptualization, M.A. and M.S.; methodology, M.A., S.P.B.C.; valida-tion, M.A, A.S., and A.C.; investigation, M.A., S.S.B., S.P.B.C., A.S., L.B., M.S.; data curation, S.S.B., S.C., A.S., L.B., M.S.; writing original draft preparation M.A. and S.S.B.; writing—review and editing, S.S.B., S.P.B.C., A.S., L.B., A.C., M.S supervision, M.A. and M.S.; project administration, M.A. All authors have read and agreed to the published version of the manuscript.

Figures


Appendix A: Preliminary results: descriptive statistics of studies concerning the management of surgical and OB/GYN patients affected by COVID-19 during the first wave (February – May 2020). Guidelines and recommendations with reported rating scales.



Appendix B: Preliminary results: descriptive statistics of studies concerning the management of surgical and OB/GYN patients affected by COVID-19 during the first wave (February-May 2020). A: H-index of selected journals. The red line indicates the mean h-index for surgical and gynecological journals according to Scimago Journal & Country Ranking. B: Levels of Evidence according to Oxford Centre for Evidence-Based Medicine Levels of Evidence (March 2009). C: Articles by type. GRADE: Grading of Recommendations Assessment, Development, and Evaluation; MiChe: Mini-Checklist (rapid assessment to evaluate the quality and utility of guidelines quickly); OCEBM: Oxford Centre for Evidence-Based Medicine; SORT: Strength of Recommendation Taxonomy.

References

  1. Prezioso C, Pietropaolo V (2020) COVID 19: update of the Italian situation. J Neurovirol. 26: 834-837.
  2. Pertile D, Gallo G, Barra F, Pasculli A, Batistotti P, et al. (2020) SPIGC Working Group. The impact of COVID‑19 pandemic on surgical residency programmes in Italy: a nationwide analysis on behalf of the Italian Polyspecialistic Young Surgeons Society (SPIGC). Updates Surg. 72: 269-280.
  3. Bitonti G, Palumbo AR, Gallo C, Rania E, Saccone G, et al. (2020) Being an obstetrics and gynaecology resident during the COVID-19: Impact of the pandemic on the residency training program. Eur J Obstet Gynecol Reprod Biol. 253: 48-51.
  4. Moher D, Liberati A, Tetzlaff J, Altman DA, Prisma Group. (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med. 151: 264-269.
  5. OCEBM Levels of Evidence Working Group. The Oxford Levels of Evidence 2. Oxford Centre for Evidence-Based Medicine.
  6. Djulbegovic B, Guyatt GH. (2017) Progress in evidence-based medicine: a quarter century on. Lancet. 390: 415-423.
  7. COVID Surg Collaborative. (2021) Machine learning risk prediction of mortality for patients undergoing surgery with periopera-tive SARS-CoV-2: the COVIDSurg mortality score, Br J Surg, 108: 1274-1292.
  8. COVIDSurg Collaborative, GlobalSurg Collaborative. (2021) Timing of surgery following SARS-CoV-2 infection: an international prospective cohort study, Anesthesia 76:748-758.
  9. Mehra MR Ruschitzka F, Patel AN. (2020) Retraction-Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet. 395: 1820.

© by the Authors & Gavin Publishers. This is an Open Access Journal Article Published Under Attribution-Share Alike CC BY-SA: Creative Commons Attribution-Share Alike 4.0 International License. With this license, readers can share, distribute, download, even commercially, as long as the original source is properly cited. Read More.

Annals of Case Reports