Key messages

  • We have analyzed how newspapers from 25 countries presented COVID-19 information and found many inconsistencies in reporting epidemiological data;

  • Newspaper websites dedicated to COVID-19 failed frequently to adjust data for the population size and for the onset of pandemic when making comparisons between countries and regions.

Researchers and journalists need to work more closely together to provide more accurate information.

Introduction

Amid the COVID-19 pandemic, social media has spread massive amounts of inaccurate information daily, reaching millions of people [1, 2]. On 15 February 2020, the World Health Organization (WHO) Director-General Dr Tedros Adhanom Ghebreyesus said, “We’re not just fighting an epidemic; we’re fighting an infodemic”. Now there is an international campaign to use the traditional media as a source of trust-based secure information, such as newspapers [3]. The WHO, from before the COVID-19 pandemic, has reinforced the important role of the press in combating the spread of misinformation and translating epidemiological data to the general audience [4, 5]. Newspapers carry a great deal of responsibility as there is evidence that their reports influence the decisions of health professionals, politicians, administrators and scientists, not only the general public [6, 7].

Journalists compare numbers of cases and deaths due to the pandemic, between regions of the same country and between countries. Does the press decipher and refine epidemiological information and present it in an appropriate way to readers? Inaccurate information may lead to wrong interpretation by the readers, increasing anxiety or may limit compliance with preventive recommendations as evidenced before, in the 2009 Influenza A (H1N1) Pandemic or 2014 US Ebola communication crisis [3, 8,9,10]. In those cases, researchers found a positive correlation between amount of information received and number of protective behaviors for Influenza A Pandemic [3]. In low-income countries, where health services are limited, misinformation may lead people to search for assistance about the dangers of COVID-19 pandemic when it is not needed, or not inadequate [11]. At the time, very few publications had evaluated how newspapers are presenting epidemiological data about COVID-19; none had evaluated newspapers from around the world for a broad perspective of how mass media communicated information [12,13,14,15,16,17].

We aimed to evaluate whether major newspapers around the world are adequately comparing the epidemiological data on COVID-19 from regions within countries and across countries including adjustment for population size; adjustment of pandemic start date with a common milestone (e.g., same proportion of infected population) and presentation of a logarithmic curve. We also aimed to determine whether they are guiding readers to properly interpret the pandemic numbers, including explanation about the meaning of logarithmic curves and about the accuracy of reported cases and deaths.

Material and methods

Study design and participants

We conducted a cross-sectional study in which we evaluated the top 3 newspapers from 25 countries for the quality of presentation of comparisons of COVID-19 epidemiological data. In most of the countries, the top 3 newspapers comprised more than 50% of the country's newspaper circulation (number of newspaper copies distributed on an average day). We selected the 15 countries with the greatest numbers of deaths adjusted for population size and the 10 most populous countries that was not on an initial list of the European Center for Disease Prevention and Control (See Table 1) [18]. The reason to select the 10 most populous countries is their importance in the global population.

Table 1 List of selected countries and the respective newspapers analyzed
Full size table

We selected the newspapers with the highest 2019 circulation in each of the 25 countries in 2019, without an eminently sporting character, according to the best ranking index available (Table 1). For each newspaper, 2 authors acted as independent raters and the website that each newspaper created to report COVID-19 data. First, the raters explored the newspaper’s website and then used the Google search engine in the local language to identify the newspapers COVID-19 webpages. Where newspapers had multiple websites, we included all of them in the analysis. We resolved disagreements between the 2 raters by consensus. There were no language restrictions. We used Google Translator (CA, United States) to translate texts written in languages other than English, Spanish, and Portuguese using Google Translator.

We also evaluated the data reported by the governments of each country (such as Ministries of Health and health authorities) using the same questionnaire applied to the newspapers (explained below) to evaluate the quality of data presentation.

Data collection

We included newspapers if they presented a website dedicated to COVID-19 with at least one table, graph, or map comparing cases, deaths, or both between regions of a country or between countries. We have not reviewed individual articles. We determined the quality of data presentations based on the answers to the following questions:

  1. 1.

    Have the numbers been adjusted for population size?

  2. 2.

    Have the start dates for comparisons been adjusted for a common milestone across regions/countries (such as from a % of cases in the population)?

  3. 3.

    Has a logarithmic curve been presented?

  4. 4.

    If so, has its meaning been explained? and

  5. 5.

    Has the newspaper provided an explanation about the difference of accuracy when reporting cases and when reporting deaths?

We considered each question as positive if one table, graph, or map fulfilled the above criteria at least once in a comparison. We selected those questions based on the recommendations for describing epidemiological data (counts and rates, time, use of logarithm scale and characterization the cases) from the Centers for Disease Control and Prevention (CDC, USA) Field Epidemiology Manual [19].

We used the same process to evaluate data reported by governments of each country (Ministry of Health and health authorities). We also stratified these results according to the 2019 United Nations Human Development Index (HDI). HDI is an index to rank countries based on life expectancy, education (literacy rate, gross enrollment ratio at different levels and net attendance ratio), and per capita income indicators. A country scores a higher HDI when the lifespan is higher, the education level is higher, and the gross national income GNI (PPP) per capita is higher [20,21,22,23]. Four the four tiers, cut-off points are: HDI of less than 0.550 for low, 0.550–0.699 for medium, 0.700–0.799 for high, and 0,800 or greater for very high human development [24]. We checked the HDI rating of each country to detect important differences between newspapers. Finally, we assessed the number of website hits using web analytics from SimilarWeb [25].

Statistical analysis

We followed the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. The data were expressed as absolute and relative frequencies of compliance with the requirements. Pearson’s chi-square test was used to compare proportions, and the significance level was set at 5% [26].

Role of the funding source

This study was funded by the authors, and they declare no conflicts of interest.

Results

We evaluated a total of 75 newspapers and 25 official bodies between 23 May 2020 and 7 June 2020. At the time of analyses, we considered all data presented by the newspaper or official body since the first case of COVID-19 reported by them in each country. The list of evaluated newspapers and the ranking index used to select them in each country are shown in Table 1. Together, the websites of these newspapers received 4, 473, 396, 452 visits in May 2020, for a total of 1, 306, 866, 593 unique visitors. Of a total of 75 newspapers analyzed, 51 (68%) presented at least one graph, table, or map comparing cases and/or deaths between regions of their country or between countries on a website dedicated to COVID-19, or both, as shown in Fig. 1. All newspapers websites and official bodies were updated daily.

Fig. 1
figure 1

Flow diagram of study analysis

Full size image

A detailed description of the comparisons appears in Table 2. On average, half of the newspapers presented a comparison of cases or deaths between regions of the same country or between countries. These results were also stratified according to the HDI of each country and differences were not of great magnitude among countries with very high, high, and medium HDI. We showed in Table 3 that the quality of data varied according to the country HDI. Twenty-seven (61.4%) of 44 newspapers from very high or high HDI countries presented at least one graph, map or table with population adjustment or time adjustment or logarithmic curve. All newspapers [7] from middle or low HDI countries have not presented any type of adjustment (p = 0.009; Chi-Square Test).

Table 2 Presence of comparisons of deaths or cases in the newspapers analyzed according to their country’s human development index (HDI)
Full size table
Table 3 Quality of comparisons presented by newspapers and health authorities according to their country’s human development index (HDI)
Full size table

Quality assessment of the comparisons showed that only a minority of newspapers adjusted the data for population size in case comparisons between regions (16 adjusted [37.2%] of 43 comparisons) and between countries (10 [25.6%] of 39), and the same was true for death comparisons between regions (10 [27.3%] of 36) and between countries (10 [27%] of 37), as shown in Table 4. Table 4 also shows time adjustment that allows the analysis of curves with different start dates. Only 11 (21.6%) of the 51 newspapers that presented comparisons considered an index date for the analysis. A logarithmic curve was rarely presented, and only 5 (29.4%) of the 17 newspapers that presented a logarithmic curve for cases or deaths explained its meaning and its purpose. El País newspaper (Spain) gives a brief explanation: "The scale may be linear (when the distance from 1 to 10 is proportionally the same as 10 to 100) or logarithmic (that better represents the nature of an epidemic, when the numbers are multiplied)" [27].

Table 4 Quality of comparisons presented by newspapers and health authorities
Full size table

We also analyzed the quality of data presentations on the official websites of health authorities in each country and identified many inconsistencies in the reporting of comparisons. For example, only 4 of 17 (23.5%) official websites provided the number of deaths relative to population size when comparing their regions. Regarding the difference between cases (influenced by the testing strategy) and deaths (less influenced), only 7 (13.7%) of the 51 newspapers that presented at least one graph, table, or map of cases explained the meaning of this difference. For example, Folha de São Paulo newspaper (Brazil) explains that "The number of confirmed cases is related to the amount of testing from each country. For example, South Korea tests most of its population. In Brazil, the strategy is to test only suspected people, which can underestimate the real number of cases" [28]. Most newspapers that compared cases or deaths (72.5%) used data provided by the Johns Hopkins University (Table 5) [29].

Table 5 Data sources used by newspapers for comparisons of cases and/or deaths between countries
Full size table

Discussion

The novel COVID-19 pandemic has permeated communities around the world at unparalleled rates. So too has an infodemic of proportions never seen before. It is spreading misinformation faster and further, creating an environment of uncertainty and anxiety. [12, 13] People view the press as an important source of more in-depth, detailed, and instructive information for the population, translating technical and scientific information into plain language. [15].

We evaluated newspapers from 25 countries and found that about 70% of them compared numbers of COVID-19 cases or deaths, or both, demonstrating that these newspapers are interested in presenting epidemiological information. We also observed that interest varied little across countries, regardless of the level of development of each, suggesting that it is a general interest. Although we didn't calculate a sample size, the monthly unique visitors of the websites that we analyzed may represent up to 25.69% of the sum of the countries’ populations [18].

In the comparisons analyzed here, only 1 in every 4 newspapers adjusted the number of cases or deaths, or both for population size; that would seem to be a minimum requirement for assessing populations of varied sizes from different regions and countries. The newspapers also failed to explain the difference between cases (subject to availability of testing, types of tests, and testing policies) and deaths (more assertive). Only 13.7% of newspapers provided this information. Cases and deaths differ conceptually in their registration, mainly because of the COVID-19 testing strategies. That is, countries with broader testing policies may have more cases due to measurement bias. On 14 June 2020, Germany and Italy case numbers appeared to be very similar. Had Germany been as heavily affected by COVID-19 as Italy? The answer is no. Germany has a larger population than Italy, and its policy of initial mass testing has been much more aggressive [30]. Therefore, the most appropriate way to analyze data is to compare the number of deaths (and not just cases) according to population size. The result would be as follows: Germany with 10.6 deaths/100 000 population versus Italy with 56.8 deaths/100 000 population. Unfortunately, we found this simple error in most newspapers analyzed. A small study of 5 newspapers identified the same situation, where the staff presented numbers of COVID-19 deaths and cases without a denominator and without context, showing the emotional appeal of the headlines [14].

Similarly, time adjustment, that would allow superimposing curves mainly in comparisons between countries, was absent in most of the comparisons analyzed. Even the websites of health authorities, referenced by many newspapers, failed to meet this requirement for high-quality comparisons. Making adjustments to allow time comparisons between countries is an important part of the analysis for comparing the COVID-19 curves [31]. We emphasize that none of the newspapers from middle or low IDH countries presented population adjustment or time adjustment or logarithmic curve for comparisons of cases or deaths between regions or countries.

Because COVID-19 is a viral disease, the number of people who get infected grows exponentially and on a linear scale graph, the rate of growth keeps going up and up—the line can become almost vertical and appear to go on forever—and can create the impression that measures like social distancing aren’t working. When using a logarithm graph of COVID-19 infections, even though the numbers are still increasing, you can identify the point at which the rate of growth starts to level off when that exponential growth has decreased. This difference may affect perception and behavior of the population during COVID-19 pandemic [32].

Newspapers with large readerships may simplify COVID-19 information to cater to a more general audience unfamiliar with epidemiological data. Explaining logarithmic curves, index date for each country, and differences between cases and deaths may be confusing for many readers. However, simple measures such as population size adjustment may help readers to better understand the comparisons.

Often newspapers need to present factual information, highlighting specific points to draw public attention, and journalists have limited time to prepare the data for publication [16]. Thus, we decided to evaluate graphs presented on websites developed by newspapers, as these would be less prone to the daily pressure of producing a next issue on time.

News articles affect people's perceptions of disease transmission, resulting in different risk behaviors as evidenced before [3, 8, 9]. During the COVID-19 pandemic, newspapers reported misinformation about the origin of the disease, risk factors, use of chloroquine and ibuprofen, in addition to presentations of pandemic data that were not clear [33,34,35,36]. An increase in depression and anxiety may be generated by the COVID-19 pandemic, exacerbated by continuing exposure to information about the disease, as evidenced in other epidemics, such as the one of Ebola virus [8, 9, 37, 38].

Newspapers are important not only to the public but also to health professionals, politicians, administrators and scientists, whose decisions are also influenced by the media [6, 36, 39]. Two Cochrane reviews provided evidence that press coverage plays an important role in influencing the use of health services and the decision to adopt healthy behaviors [11, 40]. Conversely, scientists criticized the press for presenting specific information, often shallow, exaggerated and without context, highlighting only the benefits of interventions and not consistent with the medical literature. An important limitation has been evidenced in the present study: the lack of a sense of proportion to the population’s size when reporting incidence of diseases [6, 7, 16].

When added together, the reach of the newspapers analyzed in this study amounted to more than 4 billion hits in a month. The only newspaper that met all requirements during the study period was El País from Spain. That website adjusted COVID-19 cases and deaths for population size and time evolution, in addition to presenting and explaining the meaning of logarithmic curves and highlighting the difference between cases and deaths. When presenting all of these requirements, El País helps the readers to better understand the numbers and to adequately make their conclusions.

Our study has some limitations. Our sample is limited to 3 newspapers per country, and, given the difficulty in determining the total circulation of newspapers in the world, we did not perform a sample size calculation. We omitted some well-known newspapers from the analysis because they were not among the top 3 in terms of circulation in the country. Also, because there was no universal ranking index from where to select newspapers in each country, we used several sources. Our results may overestimate the quality of comparisons. For example, if a newspaper had several comparative graphs and corrected only one of them for population size, we considered that the newspaper to have met this requirement.

To our knowledge, this is the first article to broadly analyze the approach of the media to the epidemiological data on COVID-19. For decades there has been a gap between researchers and the press [41,42,43]. As researchers, we should educate the press so that they can adequately report the data, providing accurate and correct information to the population. At the same time, the press needs to be more careful with the sources of the data for science and medical issues, promoting a closer relationship between scientists and journalists. Newspapers undoubtedly play a very important role in reporting technical information to their readers, and this article is an attempt to help newspapers improve the scientific quality of their publications.

Conclusion

This study analyzed how newspapers from 25 countries presented COVID-19 information and found many inconsistences in reporting epidemiological data. We highlighted that newspapers websites dedicated to COVID-19 failed frequently to adjust data for population size and onset of the country’s pandemic when making comparisons between countries and regions. We also noted the lack of data literacy, specifically explanation to the readers about logarithmic curves and the accuracy of reported cases and deaths.