Scientific literacy and preferred resources used by Latin American medical students during the COVID-19 pandemic: A multinational survey

Introduction

In December 2019, a cluster of cases of pneumonia of unknown origin was reported in Wuhan, China and since then, the virus has spread globally. On March 11, 2020, the World Health Organization (WHO) declared coronavirus disease 19 (COVID-19) as a pandemic.¹ Researchers throughout the globe have worked arduously to rapidly fill in the gaps of knowledge on this novel disease and this has resulted in an unprecedented surge in scientific production.² However, the pressure of produce science quickly can cause an overproduction of low-quality research. Notably, before the pandemic, it was estimated that approximately 85% of research was wasted due to flaws in design, methodology or interpretation of the data, and these challenges have been augmented due to the time constraints and poor research infrastructure during the COVID-19 pandemic.^3,4

The role of medical students has been diverse throughout history, from being participatory agents in the evolution of medical curricula to being agents of promotion, prevention and treatment of diseases.^5,6 Medical students have helped during previous disasters and emergencies such as the Spanish flu outbreak of 1918 and the 1952 polio epidemic in Denmark; and they might have an important role during the COVID-19 pandemic through essential tasks such as assistance in clinical care, telemedicine, and fighting against misinformation.^7,8

To provide high-quality healthcare services, future physicians should develop abilities to critically assess research to inform evidence-based decisions during their career, including critical appraisal of scientific literature within the curriculum of medical schools and promoting involvement in research opportunities could help students develop these skills.^9–12 Notably, the percentage of student authorship in published articles has increased importantly throughout the years.¹³

Due to their role, and the unprecedent low quality scientific information, it is indispensable that medical students have enough skills to analyse scientific data. In this study, we intend to identify preferred resources used by medical students in Spanish speaking countries in Latin America to obtain information about COVID-19 and evaluate basic critical appraisal skills through a self-report online survey.

Methods

Results

There were 770 valid responses to the survey (Underlying data: Survey results-Scientific literacy and preferred resources used by Latin American medical students during COVID-19 pandemic).¹⁴ Almost two-thirds of the participants were females (n=487, 63.2%). Most of the participants were from Mexico (n=283, 36.8%) and Ecuador (n=229, 29.7%). Students equally attended private (n=388, 50.4%) or public universities (n=382, 49.6%). We also found a similar distribution in terms of completed progress in medical school: 24.0% (n=185) in the first quarter, 24.3% (n=187) in the second quarter, 24.9% (n=192) in the third quarter, and the remaining 26.8% (n=206) in the final quarter of medical school. Distribution by gender and country is shown in Table 1.

Most medical students considered that they had a low or medium level of knowledge about COVID-19 (n=539, 70.0%), while 231 participants reported an advanced or expert level (30.0%).

Overall, medical students preferred using evidence-based clinical resources (EBCRs), such as UpToDate or Medscape, to learn more about COVID-19 (n=182, 23.6%), followed by academic journals (n=171, 22.2%), and open access courses provided by entities such as the World Health Organisation (WHO) (n=141, 18.3%). Our survey responses were grouped according to the different sub-regions of Latin America (Table 2).

Regarding the type of article, medical students preferred using case reports or case series (n=218, 28.1%) to learn more about COVID-19, followed by review articles (n=168, 21.8%) (Figure 1).

Figure 1. Preferences of type of article among medical students of Americas.

The critical factor that medical students considered when reading a scientific article was most frequently the affiliation of the authors (n=366, 47.5%), followed by the journal of publication (n=176, 22.85%), the type of study (n=138, 17.92%), and the publication status (preprint vs. published) (n=44, 5.71%).

In terms of receiving news related to the COVID-19 pandemic, medical students preferred official websites of organizations such as the WHO (n=221, 28.7%), followed by scientific journals (n=185, 24.0%), and medical platforms (n=128, 16.6%) (Table 3).

We found that approximately a third of the participants correctly identified the concept of a preprint (n=265, 34.4%), while 243 students (31.6%) correctly identified the characteristic of a predatory journal. On the other hand, 553 students (71.8%) correctly identified the concept of an open-access journal.

Two-thirds of the participants (n=526, 68.3%) had a subject focused on scientific literature critical appraisal as a part of their curriculum during medical school.

We conducted chi-squared tests to identify factors that were associated with a significant difference in the proportion of students that correctly identified the concept of a preprint or predatory journals (Extended data: Frequencies of answers and results of Chi-squares performed).¹⁴ We did not find a significant difference when comparing participants in terms of self-reported level of knowledge (low/medium vs advanced/expert) in the chance of correctly identifying the concept of a preprint (df=1, χ²=0.632, P=0.456) or a predatory journal (df=1, χ²=0.435, P=0.554).

Participants from public schools correctly identified the concept of predatory journals (n=142, 37.1%) more frequently than students that attended a private medical school (n=101, 26.0%) (df=1, χ²=11.063, P<0.01); however, there was not a significant difference while identifying the concept of a preprint (df=1, χ²=2.805, P=0.094).

Additionally, we did not find significant differences in the correct identification of preprint (df=1, χ²=0.502, P=0.514) or predatory journal (df=1, χ²=0.694, P=0.406) concepts regardless of the inclusion of a scientific literature critical appraisal subject during medical school. Finally, the proportion of participants that answered correctly was not significantly different for preprints (df=3, χ²=7.124, P=0.068) and predatory journals (df=3, χ²=5.741, P=0.125) despite the progress in medical school (first-second quarter vs third-four quarter).

Discussion

Overall, most medical students perceived that their knowledge about COVID-19 was either low or medium independently of the current academic year they belonged. This differs from what could be expected in previous studies in which students from superior years were more knowledgeable on COVID-19.¹⁵ Medical students preferred using evidence-based clinical resources as their main source for learning about COVID-19. This result is consistent with previous research that shows the popularity of these resources within the medical community.^16,17 The use of evidence-based electronic resources has been associated with reduced lengths of stay and mortality rates, and it has resulted in improved performances in standardized examinations.^18,19

Most of participants preferred getting news related to the pandemic on official websites by entities such as the WHO and the Centre for Disease Control (CDC), which could be a way to combat the current misinformation outbreak (infodemic). Social media platforms provide users with immediate access to massive amounts of content and are the perfect medium to spread questionable information easily.²⁰ A study showed that fake news is easily spread through Latin American countries through social media platforms, and fact-checking could be a potential solution to fight back against the infodemic.²¹

Notably, participants mentioned a preference for case series/reports to learn more about COVID-19, followed by narrative reviews and case-control studies. Only 112 participants (15%) preferred systematic literature reviews (SLRs) or metanalyses. The importance of addressing this trend relies on the level of evidence that well-conducted SLRs and metanalysis have compared to case series/reports, which has traditionally been portrayed on the top and the bottom of the evidence pyramid, respectively.^22,23 It would be expected that medical students who had exposure to a subject focused on critical appraisal skills during their formation would choose study designs higher among the evidence pyramid, however, our results did not show a significant difference. These could be partially attributed to the fact that during the start of the pandemic most of the articles published were retrospective studies or case reports focusing on patients’ characteristics.² Remarkably, we found that regardless of the progress in medical school, self-perceived level of knowledge about COVID-19, and the inclusion of a critical appraisal subject in the curriculum, most of the participants could not correctly identify the concept of a preprint or a predatory journal, in spite of the enormous surge in preprint production during this pandemic.²⁴ As well, in our study, only 44 participants (5.71%) chose the publication status (preprint vs published article) as a critical factor when reading an article. This is concerning as medical students might fail to recognize that preprints are preliminary reports that have not been peer-reviewed and might contain faulty or low-quality information.²⁵

Additionally, as medical students are more frequently involved in research, they could be potential targets for predatory journals that promise easy and rapid pathways to publish, through unethical practices, which could further contribute to the infodemic.¹³ As expected, predatory journals have exploited the uncertainty and prolonged reviewing and editing processes by multiple scholarly journals due to the pandemic to increase their profits.^26–28 A previous study conducted in the Kingdom of Saudi Arabia and New Zealand also revealed a poor understanding of medical students regarding predatory journals.²⁹

The main limitation to our study is the sample size, particularly in some of the Spanish-speaking countries in Latin America, which could be related to the short time that the survey was available or ineffective sampling strategies for some specific countries. The chain-referral sampling method employed, despite its pragmatism, is a significant limitation, as the sampling is non-random due to the referral process, in which the sample is dependent on the researchers’ contacts, which is reflected by the significantly higher proportion of participants from Mexico and Ecuador. As well, there is uncertainty of whether the sample is representative of the target population. Furthermore, participants could be susceptible to subject bias and might have elected responses that seemed more appropriate for a medical student.

This study is novel in its approach to identify sources used by Latin American medical students to learn about COVID-19 and to evaluate critical appraisal skills used to interpret new scientific information. Additionally, even though we had difficulties in sampling medical students from certain nationalities, we did recruit an equative number of medical students representing different stages of medical degree completion. Our results support the need to reinforce training in critical appraisal of scientific literature during medical school.^30,31

Conclusions

Throughout history medical students have an important role, and the COVID-19 crisis is not an exception. Our study found that most of medical students have good behaviours regarding acquire new information related to COVID-19 such as using evidence-based clinical resources to learn about COVID-19 and official websites of recognised organizations to receive news. However, they have problems in terms of correctly identifying predatory journals or preprint articles independently of the region, self-perceived level of knowledge, progress in medical school, or literature critical appraisal classes. Besides, these students prefer a study design that is not at the top of the evidence pyramid. Our findings could provide insights into the need to reinforce training in critical appraisal of scientific literature during medical school in Latin America.

Data availability