-
Clinician Experience with Telemedicine at a Safety-net Hospital Network during COVID-19:A Cross-sectional Survey
Objective. The COVID-19 pandemic prompted unprecedented expansion of telemedicine services. We sought to describe clinician experiences providing telemedicine to publiclyinsured, lowincome patients during COVID-19. Methods. Online survey of ambulatory clinicians in an urban safetynet hospital system, conducted May 28, 2020–July 14, 2020. Results. Among 311 participants (response rate 48.3%), 34.7% (n=108/311) practiced in primary/urgent care, 37.0% (n=115/311) medical specialty, and 7.7% (n=24/311) surgical clinics. A large majority (87.8%, 273/311) had conducted telephone visits, 26.0% (81/311) video. Participants reported observing both technical and nontechnical patient barriers. Clinicians reported concerns about the diagnostic safety of telephone (58.9%, 129/219) vs. video (35.3%, 24/68). However, clinician comfort with telemedicine was high for telephone (89.3%, 216/242) and for video (91.0%, 61/67), with many clinicians (92.1%, 220/239 telephone; 90.9%, 60/66 video) planning to continue telemedicine after COVID-19. Conclusions. Clinicians in a safetynet health care system report great comfort with and intention to continue telemedicine after the pandemic, despite safety concerns and patient challenges.
Telemedicine, safety-net hospitals, health care delivery, ambulatory care, vulnerable populations [End Page 220]
The coronavirus SARS-CoV-2 (COVID-19) pandemic compelled health care networks across the United States to rapidly expand telemedicine services.1,2 Telemedicine "seeks to improve a patient's health by permitting two-way, real time interactive communication between the patient, and the physician or practitioner at the distant site."3 Telemedicine provides means to conduct remote clinical visits, enabling decreased COVID-19 exposure risk. Remote approaches to clinical care are effective for the care of chronic medical and behavioral conditions4,5 and have high patient satisfaction;6,7 video visits have perceived quality equivalent to in-person visits.8
Despite their favorable attributes, many payors have not previously reimbursed telemedicine services,9 a major implementation barrier for safety-net health networks caring for Medicaid-and Medicare-covered patients.10–12 On March 27, 2020, the Coronavirus Aid, Relief, and Economic Security Act was signed into law, authorizing federally qualified health centers to furnish telehealth services to Medicare beneficiaries.13 Many state health agencies also authorized Medicaid payments for telemedicine services,14 including video and telephone visits. With these policy changes, institutions serving Medicare and Medicaid populations expanded their use of telemedicine.15 For example, New York City Health + Hospitals went from 500 billable televisits in February 2020 to nearly 83,000 in April 2020;16 the Ohio State University Wexner Medical Center's family medicine programs' telemedicine volume grew from 4% to 92.5%.2
The urgency of the pandemic and shelter-in-place orders, along with shifts in reimbursement policy, propelled safety-net site clinicians to rapidly adapt to provide telemedicine care, often with little advance preparation or prior experience with this care modality. Clinicians are front-line stakeholders of telemedicine care and are key determinants of its implementation.17 As social distancing protocols and the need for telemedicine persist, clinician experience will be a key factor in ensuring overall care quality and outcomes.18 However, little research has queried clinician telemedicine experience in the safety net.
Safety-net care settings serving low-income, ethnically diverse populations are unique. Patients from vulnerable populations can benefit from some of the advantages of telehealth, such as reduced transportation barriers and enhanced convenience of appointments to assist patients in not missing work shifts.19 However, safety-net patients may have limited English proficiency (LEP), limited technology literacy, and barriers accessing smartphones and cellular or broadband service.20–23 These barriers may make equitable telemedicine implementation particularly challenging in the LEP safety net.24
We sought to understand clinician experience in the safety net during implementation of telemedicine in the early phase of the COVID-19 pandemic. Highlighting frontline perspectives will be key for advancing telemedicine quality for safety-net populations, as well as long-term telemedicine sustainability.
Methods
Study setting
Our setting is a large, urban safety-net hospital network, serving publicly-insured and low-income county residents. Our patient population is ethnically and linguistically diverse, with varying health literacy25,26 and digital literacy levels27 (see Table 1 for detailed patient population demographic characteristics). Of county [End Page 221]
PATIENT DEMOGRAPHICS OF STUDY SETTING
residents, 42.3% speak a language other than English at home and 19.1% have limited English proficiency.28
Our study population included clinicians (physicians, nurse midwives, nurse practitioners, physician assistants, behavioral health clinicians, and others) providing ambulatory care services in the health care network. We excluded residents and Accreditation Council for Graduate Medical Education fellows, as they rotate through different health care delivery systems and have a different telemedicine experience from staff and faculty clinicians.
The network officially began providing telemedicine care, primarily by telephone, on March 3, 2020, anticipating COVID-19 precautions. Prior to this date, telephone and video clinical visits were performed sporadically, depending on patient preference and clinician initiative. The official shelter-in-place mandate for the county began on March 17, 2020. An official electronic health record video encounter type and documentation workflow became available on June 15, 2020. The video visit software [End Page 222] endorsed by the network was Zoom (San Jose, California). Patients were required to download the Zoom mobile application on a mobile phone or tablet, or open a link on their computer device, and would receive a meeting ID from the clinical team to begin a video visit encounter. Zoom is usable by Windows, Apple, iOS, and Android devices. Other clinicians informally used Doximity (San Francisco, California) or other personal social media video platforms, although this was not sanctioned by the network. Each clinic received basic, introductory training materials on video visits using Zoom, and clinician-and patient-facing websites to provide standardized guidance and to support onboarding. However, no system-wide infrastructure or device purchase occurred, and no system-wide standard work flow was implemented.
Outcomes and data collection
We conducted an online, anonymous quality-improvement survey through Qualtrics software (Provo, Utah), provided via a secure, university-based database (see online Appendix for full survey). We compiled validated instrument questions adapted from the Consolidated Framework for Implementation Research (CFIR),28 pre-existing telemedicine assessment tools,29–31 and front-line stake-holder recommendations. Study outcomes were to assess observed patient barriers (CFIR construct: outer setting, patient needs32), satisfaction and comfort with telemedicine,33 and intention to use telemedicine in the future by choice.30 We also developed a novel item to assess perceptions of telemedicine safety: "Compared with in-person visits, I'm concerned about the safety of telemedicine because of increased risk of missed or delayed diagnosis." Survey items were Likert-scale with optional free-text comments asking what had gone well with the telemedicine visits, what had not gone well, and option to share additional perspectives and comments. As this survey was to inform quality improvement, the study was exempt from institutional review board approval.34 We distributed the survey via email, and department-specific study champions provided reminders and encouragements to participate. We collected responses from May 28, 2020 to July 14, 2020.
Analysis
We dichotomized Likert-scale questions into binary variables. For example, for the item assessing clinician comfort providing telemedicine, we combined "very comfortable" and "somewhat comfortable" into "comfortable". To assess if outcomes differed by clinician specialty, we combined specialties into "primary/urgent care," "medical specialty," and "surgical specialty." We conducted cross-tabulations of key metrics and chi-squared analysis of dichotomized variables across clinician specialties using Stata 13.1 (College Station, Texas). We reviewed free-text comments and provided exemplars based on a narrative review of free-text comments (led by AES). Our study rationale was to provide a clear picture of the behaviors and perceptions of clinicians across specialty. Because it was cross-sectional, and there are few prior data on clinician telemedicine perceptions to guide a multivariate analysis, we elected to report key outcomes and specialty-based comparisons only. Because we were interested in clinicians' direct experiences with these modalities rather than their beliefs or perceptions, we restricted analysis of telephone visits to those who reported on average at least one telephone visit per half-day clinic session in the prior month, and restricted analysis of video visits to providers who reported conducting at least one video visit on average per half-day session in the prior month. [End Page 223]
Results
We had 311 final respondents, out of an eligible 643, for a response rate of 48.3%. Among participants, 37.0% (n=115) were medical specialists, 34.7% (n=108) were primary or urgent care clinicians, and 7.7% (n=24) were surgical specialists. Demographic characteristics of survey participants are listed in Table 2. A majority (57.2%, n=178) conducted four or more telephone visits per half-day session. Only 26.0% (n=81) conducted one or more video visits per half-day session. Below, we report summary statistics of key outcomes as well as exemplar quotations in italics from free-text responses that illustrate results. Additional free-text quotations are available in Box 1. There were no statistically significant differences in key outcome measures by clinician specialty.
We asked clinicians to select all applicable patient challenges that they had directly observed when conducting telemedicine visits over telephone and video (see Figure 1 for compilation of technical and non-technical barriers). The most common barriers for telephone visits included: speech, hearing, or cognitive barriers (44.1%; example: "Patients that are hard of hearing are almost impossible to communicate with, very challenging to reach some patients by phone."), communication quality (43.7%), and lack of having a phone (37.6%). The most common barriers for video visits included trouble using mobile applications (39.5%), lack of video (38.6%), lack of knowledge or skills to participate in the visit (37.9%; example: "Difficulty setting up video access due to language and educational barriers."), and lack of Internet (35.0%).
The majority of clinicians (129/219, 58.9%, see Figure 2) conducting telephone visits agreed that they had concerns about the safety of telephone visits, with regard to a missed or delayed diagnosis. However, only 35.3% (24/68) of clinicians doing video visits had diagnostic safety concerns. Diagnostic safety concerns related to inability to gain objective vital signs or physical exam findings: "There are patients, particularly those with heart failure, who just physically need to be seen to evaluate them (volume status in particular). I'm really flying blind, and though I'm trying my best, there's a good chance I'll be wrong and cause harm."
Despite patient challenges and safety concerns, clinician comfort with telemedicine services was high (Figure 3). The majority (216/242, 89.3%) expressed being comfortable with telephone visits: "I think there is a lot we can do over the phone. It was a good way to show patients that we care about them and believe in the importance of sheltering in place. There is a lot that absolutely works over the phone." For video visits, 91.0% (61/67) endorsed comfort conducting video visits. Along with comfort, satisfaction was also high; 72.3% (47/65) were satisfied with telephone visits and 89.2% (58/65) with video visits (Figure 4): "Providing information for results and following up on an in-person visit is satisfying to do over telephone."
Clinicians predicted high likelihood of continuing telemedicine by choice once clinic operations had returned to normal (see Figure 5); 92.1% (220/239) for telephone visits and 90.1% (60/66) for video visits: "I would love to be able to continue telemedicine but it depends on the department of public health's decisions about our clinic. It saves patients and staff lot of time and hassle (commuting, childcare, parking). I think it's great for visits that don't need an exam." [End Page 224]
DEMOGRAPHICS OF CLINICIAN SURVEY RESPONDENTS
[End Page 226]
[End Page 228]
Clinician-observed patient barriers to telemedicine care in the safety-net (a) technical challenges and (b) non-technical challenges (N=311).a
Note:
aPercentages are proportion of total survey participants (N=311) who indicated they had observed these challenges.
Discussion
To our knowledge, ours is the first assessment of the early experience implementing telemedicine across a multispecialty network of safety-net clinicians during COVID-19. Prior to COVID, assessments of clinician perceptions of telemedicine found high satisfaction and equivalent quality with in-person visits.8 Other surveys found clinician resistance to telemedicine implementation due to reimbursement, liability, and technical concerns.17 Since the pandemic, a survey of gastroenterologists found 88% rated [End Page 229]
Clinician safety concerns regarding telemedicine by specialty.
video visits as better than or as good as face-to-face, with only 41% rating telephone visits as better than or as good as in-person visits.35 Another post-COVID study found clinician satisfaction with telemedicine was correlated with stated likelihood to continue telemedicine; other predictors were perceived ease of conducting a telemedicine physical exam and a flexible personality style.36 A study of primarily medical specialties found high likelihood to continue telemedicine but lower satisfaction with telephone visits than video.37 These prior surveys have not highlighted the safety net, nor safety net-specific patient barriers.
The telemedicine explosion of rapid implementation, without dedicated resources or standard work, is common to many safety-net settings that disproportionately care for low-income populations.1,28,38 Consistent with prior studies, we found that telemedicine implementation for a lower-resource patient population has been delivered primarily [End Page 230]
Clinician comfort with providing telemedicine care by specialty.
by telephone, although video visits are being piloted.16,39 This pattern is likely related to the higher complexity of technical infrastructure and clinic workflows required to support safety-net patients to successfully participate in video visits compared with phone visits. In the absence of team-based virtual workflows for video visits, particularly at lower-resourced health care sites, clinicians are required to instruct patients on how to install and use video software during their visits, cutting into the time to provide medical care. Best practices for transforming the health care systems, such as the use of standard work and teams, should be brought to bear on telemedicine practice to enable video visit implementation.28
Both technical and non-technical challenges faced by patients in the safety net may contribute to clinician concerns about reduced access for underserved patients. [End Page 231]
Clinician satisfaction with telemedicine care compared with in-person visits by specialty.
Clinicians raised equity-related concerns about patient speech, hearing, or cogn30ive ability, access to a reliable phone number, video visit capacity at home (which requires a smartphone and Internet), reliable access to language interpreters, and patient education regarding expectations for a telemedicine visit. These barriers are closely linked to health care disparities based on disability, socioeconomic status, and educational attainment, and merit close attention for equitable implementation of telemedicine.25,41–44
We can extrapolate strategies to overcome such barriers from research on increasing use of online patient portals for vulnerable populations as well as telemedicine literature. For barriers accessing devices, programs to improve device access increase portal utilization45 and are recommended for telemedicine.46 For barriers accessing data, low-cost or free access to high-quality broadband is a key component of telemedicine [End Page 232]
Clinician reported likelihood to continue telemedicine after COVID-19 by specialty.
equity given neighborhood-based disparities.22,47 For barriers in digital literacy, robust evidence exists for interventions to support individual-level patient preparation for portal access, which is likely to extend to telemedicine as well.48–52 For limited English proficiency patients, dissemination of best practices for interpreter services must be standardized.24 Finally, the design of telemedicine platforms and instructional tools must engage patients and stakeholders from vulnerable populations, applying user-centered design or other participatory methods.53 Our group has created an online toolkit of examples and resources to support safety-net institutions seeking to overcome some of these barriers.54 [End Page 233]
Over half of respondents had concerns about the diagnostic safety of telephone visits. Free-text comments shared the concerns of relying on a telephone visit when objective vital signs or exam findings are required, considering diagnoses of higher urgency such as acute abdominal pain, or privacy concerns limiting diagnostic information gathered. This can be compounded by language barriers or limited health literacy, particularly among patients in the safety net. The current evidence base suggests that telemedicine video encounters have diagnostic accuracy roughly equivalent to in-person visits,55,56 but it is unknown if this holds for telephone-based encounters or for safety-net populations. Additional resources to train providers on how to build diagnostic confidence via telemedicine are needed. Payor reimbursement for home self-monitoring devices, such as home blood pressure monitors, scales, and pulse oximeters for qualifying co-morbidities, is another strategy to improve clinician confidence in safe, remote diagnosis and patient self-monitoring.57
Despite safety concerns and challenges, there was great interest in continuing to use telemedicine in the future by choice, after the COVID-19 pandemic has subsided. Our interpretation of the high likelihood to continue, despite safety concerns, is that the overall benefits of patient and staff convenience and satisfaction outweigh the potential risks. This is a promising finding, signifying that stakeholders are engaged in championing this service long-term. Since many state Medicaid programs model their Medicaid policy after Medicare, sustained Medicare telemedicine policy is key to shifting Medicaid nationwide. Moreover, as many safety-net patients do not have reliable video access, reimbursement for telephone-based visits will continue to be a priority to prevent widening the digital divide. Medicaid, Medicare, and other safety-net insurers must establish reimbursement policies that facilitate long-term financial sustainability (or incentives) for provision of telemedicine.58
Limitations
While our response rate was 48.3%, we had anticipated a 30–40% response rate for a non-compensated, voluntary survey conducted during a time of crisis; our response rate surpassed our expectations given this context. Clinicians who chose to participate in the survey may differ from those who did not; for example, participants with more enthusiasm about telemedicine may have elected to participate in the survey. We lack detailed demographic data for all eligible clinicians in the network to compare responders with non-responders. We do not have data on whether patient characteristics, including language preference, differed by type of telemedicine visit or clinical specialty type. We do not know if factors such as amount of interpreter use or prior telemedicine experience may have affected some of the clinician ratings of their experience conducting telemedicine. Some respondents did not complete every question of this survey, and many did not disclose their specialty, leading to varying total denominators for each survey item and clinical specialty subgroup. Based on the survey structure, there may have been participants with less than one telemedicine visit per session on average (which were categorized as zero); our results are therefore generalizable only to clinicians who conduct at least one telemedicine encounter per half-day or more. Finally, as telemedicine was not widely implemented prior to COVID-19, we could not compare changes in utilization or perceptions pre-and post-pandemic. However, study strengths include a diverse range of clinician participants who practice in a safety-net setting and our ability to capture front line perspectives at a health care site relatively early (in the first six months) in telemedicine implementation. [End Page 234]
Conclusions
Our survey describes the realities of the rapidly changing ambulatory landscape during COVID-19. Safety-net clinicians are facing multiple patient barriers to engagement in telemedicine care during this first wave of the COVID-19 pandemic. However, they also appreciate the benefits for their patients and attest high interest in continuing telemedicine, despite concerns about safety and equitable access. Safety-net patients require access to devices, data, interpreter services, and technical support in order to participate equitably in telemedicine. Provider-level supports are needed to ensure sustainability, promote safety of care, and improve satisfaction to prevent long-term burnout or adverse care outcomes. Best practices tailored for the safety net will be key to building capacity if telemedicine is to remain for the long term.
Supplemental Material
ANJANA E. SHARMA, is affiliated with the Center for Excellence in Primary Care, Dept of Family and Community Medicine, UCSF (University of California San Francisco) School of Medicine and the UCSF Center for Vulnerable Populations at Zuckerberg San Francisco General Hospital (CVP). ELAINE C. KHOONG, COURTNEY R. LYLES, URMIMALA SARKAR, and DELPHINE TUOT are affiliated with the Division of General Internal Medicine, UCSF School of Medicine, and CVP. MALINI A. NIJAGAL is affiliated with the Department of Obstetrics, Gynecology and Reproductive Sciences, ZSFG and UCSF. GEORGE SU is affiliated with the Division of Pulmonary and Critical Care, ZSFG and the Department of Medicine, UCSF School of Medicine.
Acknowledgments
The authors thank the clinicians who took the time to complete this survey. We greatly thank Kristan Olazo for assistance with image and reference formatting. We thank Christina Yong for proofreading the manuscript.
Grant Support
Author 1 (Dr. Sharma) is supported by the National Center for Advancing Translational Sciences of the NIH under Award Number KL2TR001870.
Author 2 (Dr. Khoong) is supported by the National Heart Lung and Blood Institute of the NIH under Award Number K12HL138046 and National Center for Advancing Translational Sciences of the NIH under Award Number KL2TR001870.
Authors 3 and 4 (Dr. Hijagal and Dr. Lyles) are supported by a grant from the Donaghue Foundation's Greater Value Portfolio.
Author 5 (Dr. Su) is supported by NIH R01 DK117953 and NIH R01 HL143366-01A1 and P0051172 from the California HealthCare Foundation.
Author 7 (Dr. Sarkar) is supported by NIH/NCI K24CA212294.
Author 8's (Dr. Tuot) work is partially supported by the Connected Care Accelerator Program, grant #TF2007-09373 from the Center for Care Innovations, Tides Foundation and California Health Care Foundation.
Authors 1, 4, and 7 (Dr. Sharma, Dr. Lyles, and Dr. Sarkar) are supported by the Commonwealth Fund (20202842).
This manuscript's contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH or the Commonwealth Fund. The authors have no other disclosures or conflicts of interest to declare.
