Skip to main content

ORIGINAL RESEARCH article

Front. Endocrinol., 09 November 2022
Sec. Clinical Diabetes
This article is part of the Research Topic Covid-19 and Diabetes, volume II View all 31 articles

Metformin use and mortality and length of stay among hospitalized patients with type 2 diabetes and COVID-19: A multiracial, multiethnic, urban observational study

Emily Miao&#x;Emily Miao1†Kaleena Zhang&#x;Kaleena Zhang1†Jianyou LiuJianyou Liu2Juan LinJuan Lin2Donna YooDonna Yoo1Claudene J. George*Claudene J. George3*
  • 1Albert Einstein College of Medicine Bronx, New York, NY, United States
  • 2Department of Epidemiology and Population Health, Albert Einstein College of Medicine Bronx, New York, NY, United States
  • 3Montefiore Medical Center, Division of Geriatrics, Albert Einstein College of Medicine Bronx, New York, NY, United States

Introduction: Diabetes mellitus is a common comorbidity among patients with coronavirus disease 2019 (COVID-19). Diabetic patients with COVID-19 have a two-fold increased risk of death and tend to have more severe infection compared to the general population. Metformin, a first-line medication for diabetes management, has anti-inflammatory and immunomodulatory effects. Previous studies focusing on metformin and COVID-19 clinical outcomes have had mixed results, with some showing a mortality benefit or decreased complications with metformin use. To date, few studies have analyzed such outcomes among a diverse, multiracial community.

Methods: This was a retrospective review of patients with Type 2 diabetes and a confirmed COVID-19 infection admitted to an urban academic medical center from January 1, 2020 to May 7, 2020. Baseline characteristics were collected. The primary outcomes of the study were in-hospital mortality and length of stay (LOS).

Results: A total of 4462 patients with Type 2 diabetes and confirmed COVID-19 were identified. 41.3% were Black, and 41.5% were Hispanic. There were 1021 patients in the metformin group and 3441 in the non-metformin group. Of note, more participants in the metformin group had comorbid disease and/or advanced diabetes. We found no statistically significant differences between the metformin and non-metformin group in in-hospital mortality (28.1% vs 25.3%, P=0.08) or length of hospital stay in days (7.3 vs. 7.5, P=0.59), even after matching patients on various factors (29.3% vs. 29.6%, P=0.87; 7.7 vs. 8.1, P=0.23).

Conclusion: While patients had more comorbid disease and advanced diabetes in the metformin group, there were no significant differences with regard to in-hospital mortality or length of stay due to COVID-19 compared to the non-metformin group. Prospective studies are needed to determine if there is clinical benefit for initiating, continuing, or re-initiating metformin in patients hospitalized with COVID-19.

Introduction

Diabetes mellitus is a common comorbidity among patients with coronavirus disease 2019 (COVID-19). Diabetic patients with COVID-19 have a two-fold increased risk of death and tend to have more severe infection compared to the general population (1, 2). Early case-control studies have shown that COVID-19 infected patients with diabetes had a higher risk of severe pneumonia, as well as elevated tissue injury and inflammatory markers (3, 4). Hyperglycemia is known to interfere with control of viremia and inflammation, predisposing patients to poorer outcomes (3, 5).

The apparent association of glucose metabolism with COVID-19 prognosis calls for a better understanding of anti-diabetic agents that affect blood glucose control. Metformin is a widely used first-line agent for the management of diabetes with anti-inflammatory and immunomodulatory effects (6). It has been shown to decrease TNF alpha and IL-6, which are both proinflammatory adipokines associated with visceral obesity and also implicated in morbidity from COVID-19 infection (6). There have been systematic reviews examining metformin and COVID-19 outcomes (7, 8). Some studies show a mortality benefit (9, 10), while others show no statistically significant association (11, 12). Bramante et al., 2020 found an association of metformin with decreased mortality in women alone, possibly due to sex-specific reduction of TNF alpha (6). Jiang et al., 2020 did not find an effect on 30 day all-cause mortality but found a lower incidence of acute respiratory distress syndrome (ARDS), especially in women (13). A study by Gao et al., 2020 found that metformin users had a higher risk of severe COVID-19 illness, with severity defined in terms of life-threatening complications like ARDS, septic shock, and organ dysfunction requiring ICU admission (14).

In 2010, non-Hispanic Black populations were found to have the highest prevalence of diabetes at 12.6% (15). The racial and ethnic disparities in diabetes morbidity and mortality parallel those observed during the COVID-19 pandemic. A systematic review by Mackey et al., 2020 found with moderate-to-high strength evidence that Black and Hispanic populations experienced higher rates of COVID-19 infection, hospitalization, and mortality compared to non-Hispanic Whites (16). Therefore, it is important to reproduce the studies of COVID-19 and metformin use in a multiracial, multiethnic population. We conducted a retrospective study to compare in-hospital mortality and length-of-stay (LOS) at a major academic center in the Bronx, New York.

Materials and methods

Study design and participants

This is a retrospective analysis of patients with type II diabetes and a confirmed COVID-19 infection admitted to an urban academic medical center. COVID-19 was confirmed through polymerase chain reaction (PCR) testing. The study included patients admitted from January 1, 2020 to May 7, 2020 and was approved by the institution’s institutional review board. Baseline characteristics including age, gender, race, ethnicity, comorbidities including hypertension, peripheral artery disease, coronary artery disease, chronic kidney disease, as well as number and type of anti-diabetic medications were collected. Additional data including HbA1c, anion gap at time of admission, creatinine, estimated glomerular filtration rate (eGFR), and body mass index (BMI) were also collected. The primary outcomes of the study were in-hospital mortality and length of stay (LOS).

Statistical analysis

Statistical analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC, USA). Patient demographics and clinical characteristics were summarized for the whole sample and compared between metformin and non-metformin groups using chi-squared or Fisher’s exact tests for categorical variables, and 2-sample t-tests for continuous variables. Univariate analysis was performed to assess the association between in-hospital mortality and length of stay with metformin treatment using chi-squared and 2-sample t-tests, respectively. Propensity score matching was performed on selected demographic and clinical variables by implementing a 1:1 matching algorithm, and the two outcomes were analyzed similarly using the matched data. Differences with a two-sided p-value less than 0.05 were considered to be statistically significant.

Results

Patient characteristics

A total of 4462 patients were identified. Patient demographics are shown in Table 1. The sample was 46.9% female, with a mean age (SD) of 64.4 (±16.2) years, 41.3% were Black and 41.5% were Hispanic. There were 1021 patients in the metformin group and 3441 in the non-metformin group. The prevalence of co-morbidities was higher in the metformin group when compared to the non-metformin group in such conditions as hypertension (58.2% vs. 37.5%, P<0.0001), coronary artery disease (13.4% vs. 6.7%, P<0.0001), and peripheral artery disease (1.7% vs. 1.0%, P=0.06). BMI (30.45 vs. 29.36, P<0.0001), HbA1c (8.21 vs. 6.52, P<0.0001) creatinine (1.62 vs. 2.14, P<0.0001), anion gap (17.06 vs. 16.62, P<0.01) and mean number of anti-diabetic agents (1.2 vs. 0.22, <0.0001) were also significantly higher in the metformin group. Patients in the metformin group were also significantly more likely to be on concomitant anti-diabetic agents including sulfonylureas (29.5% vs. 3.6%), dipeptidyl-4 inhibitors (36.4% vs. 4.6%), glucagon-like peptides (11.6% vs. 1.7%), sodium-glucose transport-2 inhibitors (7.0% vs. 0.4%), and insulin (36.6% vs. 11.8%) when compared to the non-metformin group (P<0.0001).

TABLE 1
www.frontiersin.org

Table 1 Patient demographics.

Outcomes

Overall, there were no statistically significant differences in in-hospital mortality or length of stay (LOS) between metformin and non-metformin groups. Results from univariate analysis and propensity score matching are shown in Table 2. Clinical outcomes of in-hospital mortality (28.1% vs 25.3%, P=0.08) and LOS in days (7.3 vs. 7.5, P=0.59) showed no significant difference between metformin and non-metformin groups. Similar analysis was completed after matching patients in the metformin and non-metformin group on factors noted to be significantly different between the two groups (Table 1). Using propensity score matching, participants were matched on factors including age, race, hypertension, coronary artery disease, insulin use, HbA1c, creatinine, BMI, and total number of anti-diabetic medications (not shown). Analysis of matched groups still showed no observable differences in either endpoint of mortality (29.3% vs. 29.6%, P=0.87) or LOS (7.7 vs. 8.1, P=0.23).

TABLE 2
www.frontiersin.org

Table 2 COVID-19 Clinical outcomes and metformin use.

Discussion

While metformin may theoretically reduce inflammatory cytokines that are implicated in COVID-19 progression, there are mixed results regarding whether metformin decreases COVID-19 morbidity and mortality in diabetic patients. To our knowledge, this is one of few studies describing metformin use and COVID-19 outcomes in an urban, multiracial, multiethnic community.

We did not observe differences in in-hospital mortality or length of stay between the metformin and non-metformin groups. Additionally, propensity score matching analysis did not show a statistically significant difference in mortality between the groups. Chronic kidney disease (CKD) burden was comparable between groups, indicating that the non-metformin group did not have significant contraindications to being prescribed metformin. We noted a mean HbA1c of 6.52% (SD 1.79) in the non-metformin group, which suggests that some patients may initially have been diagnosed with diabetes, but lowered their HbA1c through successful treatment. Patients at the cutoff point with newly diagnosed diabetes may also have opted for aggressive diet and exercise rather than pharmacologic modalities. Additionally, despite metformin being a first-line agent for the treatment for diabetes, a significant proportion (n=3441) of patients were not on metformin. This could have been due to renal impairment, the intolerability of metformin’s gastrointestinal side effects, patients being lost to outpatient follow-up after the initial diabetes diagnosis, or failure to pick up newly dispensed medications. Furthermore, while other studies have noted mortality benefits in women alone (6), we did not identify such gender-specific differences.

Of note, the metformin and non-metformin groups were imbalanced. The metformin group was more likely to have a higher comorbidity burden including hypertension (HTN), coronary artery disease (CAD), and peripheral artery disease (PAD). They had a higher mean body mass index (BMI), mean HbA1c, and mean number of anti-diabetic agents including insulin. Many of these conditions, including HTN, obesity, and cardiovascular disease have known associations with poorer COVID-19 outcomes (17). In contrast, the non-metformin group had a mean HbA1c close to the lower limit of a diagnosis of diabetes, suggesting that patients early on in the diagnosis or that have been successfully treated are disproportionately represented. Therefore, a mortality benefit from metformin, if any, may have been masked by a metformin group that had far more comorbid disease and advanced diabetes. While difficult to ascertain, metformin may even have played a role in offsetting the severity of COVID-19 disease and, subsequently, preventing patients with diabetes and COVID-19 from being admitted and hospitalized, being treated at home, and thus excluded from this study. Furthermore, it is possible that the metformin group would have had a higher mortality and a longer in-hospital length of stay, but did not due to the benefits of metformin.

There may be evidence that the overall health of the metformin group may be related to a detectable difference in study outcomes. The CORONADO study was a large European observational study (n = 2449) that evaluated mortality and disease complications among patients with diabetes hospitalized with COVID-19 (18). In contrast to our study, their metformin group was overall younger and more racially and ethnically homogenous. They had a shorter duration of diabetes, fewer diabetic complications, and lower rates of comorbidities compared to the non-metformin group. They were more often using other oral agents; insulin therapy was two times less prevalent. The CORONADO study found that metformin users had significantly lower mortality rates on day 7 (8.2% vs. 16.1%, P<0.0001) and day 28 (16.0% vs. 28.6%, P<0.0001) compared to the non-metformin population (18).

In comparison, the TOGETHER trial (n = 418) is the first known randomized controlled trial to evaluate metformin in reducing time to hospitalization and mortality in an ambulatory setting (19). Symptomatic patients with a positive COVID-19 test were randomized to metformin vs. placebo for 10 days. The patient characteristics of the two groups were generally balanced with respect to age, BMI, and co-morbidities. Ultimately, the study did not find that metformin had a clinical benefit in the outpatient setting, and did not recommend repurposing of metformin as early treatment for COVID-19 disease (19). Further studies can examine metformin in COVID-19 recovered patients to assess when to re-initiate treatment. To evaluate the protective mechanism of metformin, these studies may choose to trend objective signs of disease recovery including inflammatory markers or viral load.

Most studies evaluating metformin use and COVID-19 clinical course have been conducted outside of the United States (U.S.), and fewer have evaluated clinical outcomes among a racially and ethnically diverse patient population (9). To the best of our knowledge, the only other study in the U.S. that has characterized patient demographics and clinical outcomes in an urban cohort was conducted by Crouse et al., 2020 (9). Although the study authors found metformin use to be associated with reduced mortality in patients with COVID-19 and diabetes, their sample size of metformin users was limited (n = 76) (9). An international study in the UK by Heald et al., 2022 found social disadvantage, as assessed by the Townsend score, to be associated with increased mortality from COVID-19 independent of diabetes status, and metformin use with decreased mortality (20). The intersectionality of race and economic status, however, calls for further consideration of such variables in a study population and healthcare setting more reflective of those communities with high disease burden in the U.S.

There are several strengths and limitations to this study. Strengths of this study include a large sample size and a focus on a highly diverse, at-risk population that is underrepresented in scientific literature. Given that Black and Hispanic populations experienced higher rates of COVID-19 infection, hospitalization, and mortality compared to non-Hispanic Whites (16), it is important to explore the utility of medications with proposed benefits in these specific populations. Our study builds a foundation for further investigations into the proposed benefits of medications that may decrease mortality in patients with diabetes and COVID-19. Our study adds to the observational work by Crouse et al., 2020, which was also conducted in a diverse urban cohort, but was limited by sample size of their metformin users (n = 76) (9). There are several limitations to our study. Due to the retrospective nature of our study, there may have been inconsistencies in provider medication reconciliation, limiting reliability of chart documentation. Information including metformin dose, formulation, adherence, and duration of use were unable to be obtained from chart review. Many variables including smoking status, diabetes duration, and mean blood glucose levels were not collected, which may have influenced the outcomes. For example, smoking is an independent risk factor for mortality in COVID-19 patients and therefore, could have been a confounding variable in our study (21).

Conclusion

While patients had more comorbid disease and advanced diabetes in the metformin group, there were no significant differences in in-hospital mortality or length of stay due to COVID-19 compared to the non-metformin group. Prospective studies in diverse populations are needed to determine if there is clinical benefit for initiating, continuing, or re-initiating metformin in patients with COVID-19.

Data availability statement

The de-identified data set is shared by individuals on the institutional IRB and is not publicly available. Requests to access these datasets should be directed to CG: email: clgeorge@montefiore.org.

Ethics statement

The studies involving human participants were reviewed and approved by Albert Einstein College of Medicine, Montefiore Medical Center Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

EM: Design, data management, interpretation of data, and preparation of manuscript; KZ: Design, data management, interpretation of data, and preparation of manuscript; JiL: Data management, analysis, interpretation of data, preparation of manuscript; JuL: Data management, analysis, interpretation of data, preparation of manuscript; DY: Interpretation of data, preparation of manuscript; CG: Design, acquisition of subjects, data management, analysis, interpretation of data, preparation of manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by a National Institute of Health/National Institute of Aging (K23 AG062807-02). The funding source has no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Apicella M, Campopiano MC, Mantuano M, Mazoni L, Coppelli A, Del Prato S. COVID-19 in people with diabetes: understanding the reasons for worse outcomes. Lancet Diabetes Endocrinol (2020) 8:782–92. doi: 10.1016/S2213-8587(20)30238-2

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Lim S, Bae JH, Kwon HS, Nauck MA. COVID-19 and diabetes mellitus: from pathophysiology to clinical management. Nat Rev Endocrinol (2021) 17:11–30. doi: 10.1038/s41574-020-00435-4

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Yang W, Sun X, Zhang J, Zhang K. The effect of metformin on mortality and severity in COVID-19 patients with diabetes mellitus. Diabetes Res Clin Pract (2021) 178:108977. doi: 10.1016/j.diabres.2021.108977

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Guo W, Li M, Dong Y, Zhou H, Zhang Z, Tian C, et al. Diabetes is a risk factor for the progression and prognosis of COVID-19. Diabetes Metab Res Rev (2020) 36(7):e3319. doi: 10.1002/dmrr.3319

CrossRef Full Text | Google Scholar

5. Zhu L, She ZG, Cheng X, Qin J, Zhang X, Cai J, et al. Association of blood glucose control and outcomes in patients with COVID-19 and pre-existing type 2 diabetes. Cell Metab (2020) 31:1068–77.e3. doi: 10.1016/j.cmet.2020.04.021

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Bramante CT, Ingraham NE, Murray TA, Marmor S, Hovertsen S, Gronski J, et al. Metformin and risk of mortality in patients hospitalised with COVID-19: a retrospective cohort analysis. Lancet Healthy Longevity (2021) 2:e34–41. doi: 10.1016/S2666-7568(20)30033-7

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Singh AK, Singh R, Saboo B, Misra A. Non-insulin anti-diabetic agents in patients with type 2 diabetes and COVID-19: A critical appraisal of literature. Diabetes Metab Syndr (2021) 15:159–67. doi: 10.1016/j.dsx.2020.12.026

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Zangiabadian M, Nejadghaderi SA, Zahmatkesh MM, Hajikhani B, Mirsaeidi M, Nasiri MJ. The efficacy and potential mechanisms of metformin in the treatment of COVID-19 in the diabetics: A systematic review. Front Endocrinol (Lausanne) (2021) 12:645194. doi: 10.3389/fendo.2021.645194

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Crouse AB, Grimes T, Li P, Might M, Ovalle F, Shalev A. Metformin use is associated with reduced mortality in a diverse population with COVID-19 and diabetes. Front Endocrinol (Lausanne) (2020) 11:600439. doi: 10.3389/fendo.2020.600439

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Luo P, Qiu L, Liu Y, Liu X, Zheng J, Xue H, et al. Metformin treatment was associated with decreased mortality in COVID-19 patients with diabetes in a retrospective analysis. Am J Trop Med Hyg (2020) 103:69–72. doi: 10.4269/ajtmh.20-0375

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Oh TK, Song I-A. Prior metformin therapy and 30-day mortality in patients with acute respiratory distress syndrome: a nationwide cohort study. Ann Palliative Med (2020) 9:903–11. doi: 10.21037/apm.2020.04.25

CrossRef Full Text | Google Scholar

12. Chen Y, Yang D, Cheng B, Chen J, Peng A, Yang C, et al. Clinical characteristics and outcomes of patients with diabetes and COVID-19 in association with glucose-lowering medication. Diabetes Care (2020) 43:1399–407. doi: 10.2337/dc20-0660

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Jiang N, Chen Z, Liu L, Yin X, Yang H, Tan X, et al. Association of metformin with mortality or ARDS in patients with COVID-19 and type 2 diabetes: A retrospective cohort study. Diabetes Res Clin Pract (2021) 173:108619. doi: 10.1016/j.diabres.2020.108619

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Gao Y, Liu T, Zhong W, Liu R, Zhou H, Huang W, et al. Risk of metformin in patients with type 2 diabetes with COVID-19: A preliminary retrospective report. Clin Transl Sci (2020) 13:1055–9. doi: 10.1111/cts.12897

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Gaskin DJ, Thorpe RJ Jr., McGinty EE, Bower K, Rohde C, Young J, et al. Disparities in diabetes: the nexus of race, poverty, and place. Am J Public Health (2014) 104:2147–55. doi: 10.2105/AJPH.2013.301420

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Mackey K, Ayers CK, Kondo KK, Saha S, Advani S, Young S, et al. Racial and ethnic disparities in COVID-19-Related infections, hospitalizations, and deaths : A systematic review. Ann Intern Med (2021) 174:362–73. doi: 10.7326/M20-6306

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Ejaz H, Alsrhani A, Zafar A, Javed H, Junaid K, Abdalla A, et al. COVID-19 and comorbidities: Deleterious impact on infected patients. J Infect Public Health (2020) 13:1833–9. doi: 10.1016/j.jiph.2020.07.014

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Lalau J-D, Al-Salameh A, Hadjadj S, Goronflot T, Wiernsperger N, Pichelin M, et al. Metformin use is associated with a reduced risk of mortality in patients with diabetes hospitalised for COVID-19. Diabetes Metab (2021) 47:101216. doi: 10.1016/j.diabet.2020.101216

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Reis G, dos Santos Moreira Silva EA, Medeiros Silva DC, Thabane L, Cruz Milagres A, Ferreira TS, et al. Effect of early treatment with metformin on risk of emergency care and hospitalization among patients with COVID-19: The TOGETHER randomized platform clinical trial. Lancet Regional Health - Americas (2022) 6:100142. doi: 10.1016/j.lana.2021.100142

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Heald AH, Jenkins DA, Williams R, Sperrin M, Mudaliar RN, Syed A, et al. Mortality in people with type 2 diabetes following SARS-CoV-2 infection: A population level analysis of potential risk factors. Diabetes Ther (2022) 13(5):1037–51. doi: 10.1007/s13300-022-01259-3

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hou HH, Li Y, Zhang P, Wu J, Shi L, Xu J, et al. Smoking is independently associated with an increased risk for COVID-19 mortality: A systematic review and meta-analaysis based on adjusted effect estimates. Nicotine Tob Res (2021) 23(11):1947–51. doi: 10.1093/ntr/ntab112

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: diabetes, COVID-19, metformin, urban, multiracial, multiethnic

Citation: Miao E, Zhang K, Liu J, Lin J, Yoo D and George CJ (2022) Metformin use and mortality and length of stay among hospitalized patients with type 2 diabetes and COVID-19: A multiracial, multiethnic, urban observational study. Front. Endocrinol. 13:1002834. doi: 10.3389/fendo.2022.1002834

Received: 25 July 2022; Accepted: 21 October 2022;
Published: 09 November 2022.

Edited by:

Åke Sjöholm, Gävle Hospital, Sweden

Reviewed by:

Fahad Aljuraibah, King Saud bin Abdulaziz University for Health Sciences, Saudi Arabia
Wenfang Xia, Huazhong University of Science and Technology, China

Copyright © 2022 Miao, Zhang, Liu, Lin, Yoo and George. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Claudene J. George, clgeorge@montefiore.org

These authors have contributed equally to this work and share first authorship

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.