Experiences with Telehealth for Outpatient Palliative Care: Findings from a Mixed-Methods Study of Patients and Providers across the United States

Abstract

Background:

The COVID-19 pandemic introduced a rapid adoption and scale-up of telehealth for palliative care services in the United Sates.

Objectives:

To examine and compare in-person versus telehealth experience among outpatient palliative care programs and patients.

Design:

Mixed-methods study (1) comparing patient experience survey data received between September 2020 and February 2021 from patients who received only in-person care versus those who received only telehealth and (2) qualitative interviews with outpatient palliative care providers. Data for this study were collected as part of a larger effort to develop quality measures for outpatient palliative care in the United States.

Setting/Subjects:

Outpatient palliative care patients and programs.

Measurements:

We measured patients' experiences of “feeling heard and understood” by their palliative care provider and team and their overall rating of their provider and team. We also conducted in-depth semistructured interviews with 47 palliative care providers across 25 outpatient palliative care programs.

Results:

Of 1753 patient experience surveys, 26% reflected telehealth only versus 74% in-person only. Patients in both groups reported highly positive experiences; there were no differences in “feeling heard and understood” or the overall ratings of the provider and team between the telehealth-only and in-person-only groups. Palliative care program leaders described the benefits and challenges of telehealth, including increased efficiency, the ability to incorporate family members, and challenges conducting a physical examination.

Conclusion:

Data from this study provide preliminary evidence of overall positive experiences of telehealth for outpatient palliative care among patients and providers; future research is needed to examine the sustainability of telehealth for palliative care.

Introduction

Telehealth for palliative care has the potential to improve and expand access to otherwise limited health care resources by reaching patients in remote and rural areas and enabling patients to receive care from the comfort of their own home without having to travel to a clinic.^1,2 Although evidence on the use and outcomes of telehealth for outpatient palliative care remains limited,^3–5 existing studies suggest that telehealth improves patient and family experiences and outcomes in both rural and urban settings^6–9 and improves pain intensity, severity, and interference.⁴

The use of telehealth for palliative care in the United States was fairly limited before the onset of COVID-19,¹ which forced a rapid adoption of virtual care services.^10–12 To support this necessary shift, Medicare provided reimbursement for telehealth visits under the 1135 emergency waiver.¹³ This change supported the increased use of telehealth for inpatient palliative care for communication among isolated patients and their families, patients and their providers, and among providers themselves during the COVID-19 pandemic, in addition to the delivery of palliative care services previously provided in outpatient settings.^10,14–16 Very early evidence suggests that this shift may have had wider positive impacts; a study underway¹⁶ at a large cancer institution demonstrated that goals-of-care conversations (GOC) increased with the transition to a telehealth model for outpatient palliative care visits during the pandemic.

Despite the potential for telehealth to improve palliative care access and the growing use of telehealth services spurred by the pandemic, several questions and challenges related to telehealth use, patient experience, and sustainability for palliative care remain. Palliative care centers on an interpersonal and a humanistic delivery of care to vulnerable patients, which may be challenged by the remote nature of telehealth. Elucidating the barriers and facilitators to telehealth expansion and how it affects patient experience can provide important guidance to palliative care programs seeking to scale up high-quality, patient-centered telehealth services in the future. The purpose of this mixed-methods study was to examine and compare telehealth experiences among outpatient palliative care programs and their patients in the United States.

Methods

Overview

We sought to (1) assess the impact of telehealth use on patient experience; specifically on the experience of “feeling heard and understood” by the palliative care provider and team; and (2) qualitatively understand the benefits and challenges of telehealth implementation by outpatient palliative care programs.

Data for the current study were collected as part of a larger national study to develop quality measures for outpatient palliative care. We collected patient experience survey data between November 2019 and February 2021, with a pause in data collection from April to August 2020 due to the COVID-19 pandemic, from patients receiving outpatient palliative care. Between November 2019 and March 2020, telehealth visits were very limited with inconsistent coding, making it difficult to accurately identify telehealth visits for study.

We used data from surveys fielded and received between September 2020 and February 2021, when many palliative care programs had established new telehealth care services or expanded existing services, to meet the needs of their patients during the pandemic. Coding of these visits was more consistent, likely due to new reimbursement policies. We also conducted in-depth semistructured interviews with program leaders, palliative care providers, and quality/data analysts from a sample of the outpatient palliative care programs participating in the study. The study was approved by RAND's IRB.

Sampling and recruitment procedures

Programs

Eligible programs were recruited for the larger study from a list of 360 palliative care programs that reported providing ambulatory, clinic-based palliative care to two separate registries maintained by the Center to Advance Palliative Care (CAPC). We purposively sampled to ensure representation based on administrative affiliation (i.e., hospice, hospital, ambulatory, and other affiliation) and by geographical location (i.e., U.S. Census regions). Of the 238 palliative care programs we contacted, 70 programs were deemed ineligible to participate in the larger study (e.g., provided no/limited outpatient palliative care), 124 declined participation or did not respond, and 44 programs agreed to participate.

Patients

We surveyed all patients aged 18 years or older within 6 months of an outpatient palliative care visit with an eligible provider from participating palliative care programs. For this study, we included the subset of patients who were fielded and returned a survey between September 2020 and February 2021.

Data collection procedures

Patient surveys

We used a mixed-mode survey administration procedure based on prior research demonstrating higher response rates for mail with telephone follow-up.¹⁷ This included a mailed prenotification letter to inform patients of the study with a link to a web-based survey,¹⁸ followed within a week by a mailed survey. If the survey was not returned within three weeks of mailing, up to eight attempts were made to contact the patient by telephone and complete the survey via a computer-assisted telephone interview.

The survey instrument was informed by the goals of the larger study to develop two quality measures of pain management and communication in ambulatory palliative care. It included questions related to individuals' care experiences with their identified palliative care provider and team in the areas of communication, pain management, emotional support, and overall care, as well as questions about their general health, mood, and cognition, and demographic characteristics.

We focused on a subset of survey items most relevant to our assessment of patient experience with telehealth (Appendix A1). This included a “feeling heard and understood” scale derived from items used in other studies.^19–24 Each item was framed as a statement and used a 5-point response scale (not at all true to completely true).

We conducted an exploratory factor analysis and item response theory analysis, combined with expert panel input, to finalize a four-item* summed scale with excellent internal consistency (Cronbach's α = 0.90) and test–retest reliability (r = 0.86). Scale validity was established by examining correlation with the CAHPS Clinician and Group Survey 3.0 four-item provider communication measure (higher scores on the scale were associated with higher CAHPS communication scores (r = 0.45, p < 0.001).²⁵ Because the observed item response distribution was skewed toward the high end of the scale (i.e., “completely true”), the scale score was calculated by first converting each item into a binary response option, where 1 = “completely true” and 0 = all other responses), and then averaging the top-box response across the four items.

We also included an overall rating of care question (0 = worst provider and team possible to 10 = best provider and team possible), questions on the type of telehealth (telephone, video, or both), whether it was easy to get care via telehealth, whether the patients got the care they wanted via telehealth (both on a Yes Definitely, Yes Somewhat, or No response scale, collapsed to binary yes/no), and how satisfied they were with their care (1 = completely satisfied, 5 = not at all satisfied), and demographic questions.

Program interviews

We randomly selected a subset of participating palliative care programs from each U.S. Census region, between May and July 2020, to participate in a 60-minute in-depth, semistructured interview. We randomly sampled participating programs based on size, administrative affiliation, and geographical location. We contacted 34 programs and conducted interviews with 25 programs (2 declined participation, 6 did not respond, 1 was scheduled but later canceled). Respondents from each program included palliative care providers and/or program leaders.

Program interviews were conducted by two researchers trained in qualitative research methods with a background in palliative and end-of-life care research. The interviews were facilitated using a semistructured protocol (Appendix A2), including six questions focused on exploring the impact of the pandemic on clinical practice and care delivery, including the shift to or increased use of telehealth. Interviews were conducted through Microsoft teams, using both the video and/or audio functions. Interviews were recorded and ranged from 24 to 63 minutes. A trained notetaker took verbatim notes from the interview recording.

Analysis

Survey data

We stratified the survey sample into patient respondents who were identified in program data files as having only telehealth visits; that is, no in-person visits (telehealth subgroup) versus those receiving only in-person care, that is, no telehealth visits (in-person subgroup). We compared respondent demographics between these two subgroups using t-tests or chi-square tests. Within the telehealth subgroup, we descriptively analyzed type of telehealth use (i.e., phone only, video only, both phone and video) and experiences with telehealth. We compared differences in “feeling heard and understood” between the telehealth and in-person subgroups using t-tests, and by mode of telehealth using a one-way analysis of variance (ANOVA). We compared differences in mean overall ratings of provider and team using t-tests. Statistical significance was set at <0.05.

Qualitative data

Interview data were analyzed simultaneously during data collection using a rapid thematic analysis facilitated through a detailed findings matrix in which key summary points from the interviews were input into the matrix. The matrix was structured by important topics for analysis (e.g., prior experience with telehealth, provider challenges with telehealth, perceived patient challenges), in which data were extracted from each interview and imported into the matrix immediately following the interview to ensure that data collection and analysis occurred simultaneously. Exemplary quotations from interviews were added to the matrix following transcription of the interviews. A rapid qualitative analysis is particularly useful during rapidly changing times or complex public health emergencies²⁶ without compromising the rigor of the analysis.^27,28 The findings from the interviews were reviewed and discussed with the team members during the data collection process.

Results

Patient experience with telehealth

Between September 2020 and February 2021, we fielded 5011 surveys to patients receiving outpatient palliative care from a participating program and received 2115 surveys (42% response rate), of which 327 were excluded from analyses because they were ineligible for the larger study (i.e., the respondent disavowed the reference provider/program, a proxy respondent completed the survey instead of the patient, or the patient was deceased when the survey arrived). Of the remaining 1788 patient respondents, 35 reported having a mix of telehealth and in-person visits during the reference time frame and were excluded from the current analysis. The remaining 1753 patient respondents were divided into the telehealth subgroup (n = 453) and the in-person subgroup (n = 1300).

Patients in the in-person subgroup were on average older (63.7 years vs. 60.1 years; p < 0.001) and more often White (88% vs. 81%, p = 0.002) than patients in the telehealth subgroup, but there were no significant differences by sex or education level (Table 1).

Table 1.

Demographics of Survey Respondents

	No telehealth (n = 1300)	Telehealth only (n = 453)	Adjusted p-value
Age, mean (SD)	63.7 (13.1)	60.1 (13.1)	<0.001^*
Sex (female), n (%)	708 (54)	260 (57)	0.427
Education (some college or more), n (%)	859 (66)	321 (71)	0.123
Race (White), n (%)	1144 (88)	368 (81)	0.002^*

Benjamini–Hochberg corrected p < 0.05.

SD, standard deviation.

Within the telehealth subgroup, 41% (n = 187) received only video visits during the reference time frame, 30% (n = 137) received both telephone and video visits, and 22% (n = 98) received only telephone visits. Information was missing from 31 respondents. Patient experiences with telehealth were generally highly positive; 96% of patients in the telehealth subgroup responded affirmatively that it was easy to use the telephone or video to get care from their palliative care provider and team, and 97% said they got the care they wanted during their telehealth visits. On average, patients were completely satisfied with the care they received during telehealth visits (mean score 1.52, standard deviation = 0.87, where 1 = completely satisfied and 5 = not at all satisfied).

The majority of patients in both subgroups felt completely heard and understood by their provider and team. Table 2 shows the percent of respondents who responded “completely true” for each of the four items comprising the Feeling Heard and Understood scale, by subgroup, and by telehealth type. When examining scale scores, there were no significant differences in “feeling heard and understood” between the in-person subgroup and the telehealth subgroup (mean score = 0.70 [0.37] vs. 0.72 [0.36]; p = 0.29). Within the telehealth-only group, there were no significant differences in “feeling heard and understood” by telehealth type (F_3,443 = 1.87, p = 0.13). There were also no differences in mean overall ratings of the provider and team between the in-person subgroup and the telehealth subgroup (mean = 9.22 [1.43] vs. 9.20 [1.65]; p = 0.77).

Table 2.

“Completely True” Responses to the Feeling Heard and Understood Items

Feeling heard and understood item	N (%)
Feeling heard and understood item	No telehealth (n = 1300), n (%)	All telehealth (n = 453) ^a , n (%)	Phone only (n = 98), n (%)	Video only (n = 187), n (%)	Phone/video (n = 137), n (%)
Q12: I felt heard and understood by this provider and team.	851 (66.5)	318 (70.8)	65 (67)	143 (76.9)	95 (69.3)
Q16: I felt this provider and team put my best interests first when making recommendations about my care.	955 (74.7)	342 (76.3)	69 (71.1)	148 (79.1)	106 (78.5)
Q18: I felt this provider and team saw me as a person, not just someone with a medical problem.	994 (77.4)	352 (78.4)	75 (77.3)	151 (80.7)	106 (77.9)
Q20: I felt this provider and team understood what is important to me in my life.	799 (62.6)	286 (64.4)	57 (60)	126 (68.1)	88 (65.2)

Respondents answer “Completely Telehealth” category.

Of the 453 respondents in the telehealth subgroup, 31 did not provide information regarding type of telehealth used.

Program experience with telehealth

We interviewed 47 respondents across 25 outpatient palliative care programs. All 25 programs implemented telehealth during the pandemic using video and/or phone. All but three programs had no or very little experience using telehealth for their outpatient palliative care practice before the pandemic. These programs with no or little experience with telehealth rapidly implemented telehealth during the pandemic, some over the course of a few days.

Palliative care program representatives described a range of experiences with telehealth. Table 3 presents the qualitative findings with corresponding quotes from respondents.

Table 3.

Perceived Benefits and Challenges from Program Interviews

Benefits
Promotes Efficiency, Productivity, Flexibility	“The clinician sometimes has more time to connect because of the efficiency of time involved in virtual visits, sometimes more so as in person. Scheduling is usually more flexible, which also creates more access. And frail patients who don't have stamina for coming in to the clinic, this is a great way to provide care.” “Telemedicine has made access to our clinic better, and our weekly number of patients seen is up.” “The genie is out of the bottle… I think there is potential to resolve some of the inequities and access issues that exist in palliative care.”
Offers “Window” into Patients' Lives	“[There is a] clinician benefit to see into the personal setting of the patient. You can see their home and feel a personal connection.” “Patients being comfortable at home allows people to be less stressed and contact us more often and more easily. we get more information from them, and we get a sneak peak into their environment.”
Facilitates GOC and Incorporates Family Members	“I've signed more DNR's during the pandemic than in the last year!” “I'm pretty surprised how comfortable I felt caring for patients, changing their meds, you know having tearful end of life discussions with them, people I've never met in person just call them up on video and 10 minutes later everyone is crying, like you do in in-person clinic sometimes. All of that goes pretty well over video.”
Challenges
Human Connection and Physical Touch	“[Telehealth is] cognitively and emotionally challenging because [providers] really have to do everything possible to feel present with patients.” “Outpatient team struggled at first, without the face to face contact, but adapted quickly.” “It's hard over a screen to do a good patient assessment, it's hard to pick up on non-verbal cues and pick up whether their pain is well controlled as they're telling you when you're looking at them on a tiny screen. “
Workflow and Technological Challenges	“[We're] struggling with Epic and My Chart. Zoom video visits are in MyChart, but most of our patients don't have MyChart, so there was facilitating for that, and some patients didn't have the technology for the video visits, but we did have a lot of success with people. Definitely a learning curve on the caregiver side.” “Even though we have resources and commitment, it takes multiple phone calls to patients: calling for scheduling, inventory to see the tech, another person calls, and a question pops up about clinical, so we have to have someone else call. Concerned of volume of calls made to patients to organize all the things we need for intake. I got to consent you, register and do all these things and it takes multiple calls, its clunky, not smooth at all. Worry about how patients and family feel about that. I wish we had a better workflow.”
GOC and ACP Discussions	“I found myself for particularly serious conversations that it was almost harder for patients to get all that information and I found myself having to bring them back into the clinic for an in person visit the following week to have that conversation in person.”

ACP, advance care planning; GOC, goals-of-care conversations.

Benefits of using telehealth for outpatient palliative care visits

Promotes efficiency, productivity, and flexibility

Many programs reported an overall increased efficiency because of a decrease in “no-show” visits and cancelations with telehealth visits. They were better able to integrate interprofessional staff (including a social worker or nurse practitioner) into the telehealth visit.

Offers a “Window into Patients' Lives”

Telehealth visits provided an opportunity to gain insight into patients' environment and their surroundings that they may not have been able to glean from an in-person visit. Some providers stated that seeing patients through video in their own homes allowed for a personal connection. A few providers described increased ease of medication reconciliation because patients were able to access their medication bottles during the visit.

Facilitates GOC and incorporates family members

Some providers described the benefits of telehealth for engaging patients in GOC around their preferences and values for care, enabling them to have deeper and more meaningful conversations through telehealth and integrate family members who may not have otherwise been able to attend an in-person visit with the patient. Some providers also expressed that the COVID-19 pandemic offered them an opportunity to raise issues with patients around code status and goals of care, as patients were thinking more about their values during the public health crisis.

Challenges using telehealth

Human connection and physical touch

Providers expressed that developing a meaningful connection with patients was often challenging through a virtual connection, particularly for patients who are vulnerable, ill, and older. Some providers described a loss of intimacy or human connection with patients, as they were unable to touch the patient or put their hand on a patient's knee to reassure them of their concerns, or conduct a physical examination. Some discussed overcoming these challenges through using different communication strategies such as emphasizing their own body language to validate patients' concerns, asking patients more questions, looking at their surroundings on the video, and observing the patient's body language.

Workflow and technological challenges

Providers and program leaders discussed some workflow technological difficulties adapting to the new technology platform using telehealth on the part of providers, but many of these challenges were overcome over time with increased familiarity or with additional provider training. Many program leaders discussed challenges for patients, particularly those who are older and sick, including not having access to the Internet, not understanding how to use the telehealth platform, or confusion due to using multiple platforms for different providers.

Communication around goals of care and advance care planning

Some program representatives described challenges in communicating or delivering “bad news” to patients through telehealth in addition to having conversations around goals of care or completing advance directives. While most respondents found telehealth to be helpful for engaging and including the patient's family members during the visits, a couple of providers mentioned challenges verifying the family members' identity through telehealth to ensure patient privacy.

Discussion

While telehealth is relatively new to palliative care, recent policy and reimbursement changes driven by the pandemic suggest that it is likely to endure. Our study suggests that palliative care outpatients have equally positive experiences with their palliative care provider and team whether in-person or via telehealth. Moreover, our data suggest that although outpatient palliative care programs faced some initial challenges in their adoption of telehealth due to the public health crisis, there were able to quickly adapt and implement patient-centered telehealth services. Taken together, these findings provide strong preliminary support for continued implementation and expansion of telehealth services for outpatient palliative care.

Our findings mirror prior research in other health care settings and among diverse populations, indicating high levels of satisfaction with telehealth.^29–32 Benefits cited include increased access and decreased travel time, lower costs, and better communication, all of which are likely to be relevant to telehealth for palliative care. Many known barriers to telehealth³¹; for example, Internet speed/signal coverage, technological incompatibilities, and costs, are also likely to be similar; however, there may be barriers unique to palliative care delivery that warrant further study. These include difficult expressing and sharing emotion, patients' illness acuity and cognitive and sensory impairments, physician communication, especially around difficult topics such as prognosis, and privacy issues.

Still, telehealth for palliative care may be uniquely positioned to provide services virtually because of its focus on establishing trust and strong interpersonal relationships as foundational to high-quality clinical care. Indeed, we found no differences in the patient-reported experiences of “feeling heard and understood” by their outpatient palliative care provider and team, or in their overall rating of their provider and team, when care was provided virtually or in-person. Despite initial concerns from palliative care programs about the lack of human connection via telehealth, there was little variation in the positive experience reported by patients, underscoring how well palliative care performs overall in the interpersonal domains of care. Future work could elucidate the specific processes and structures that facilitate successful tele-palliative care to establish best practices.

While some palliative care programs expressed hope that telehealth will improve access to palliative care given workforce shortages in palliative care,^2,33 it is possible that telehealth could exacerbate inequal access to palliative care for some already-disadvantaged populations. An early report of telehealth during the COVID-19 pandemic from one large health system suggests differences in access to telehealth by race/ethnicity, sex, age, language, and socioeconomic status.³⁴ Future research should also consider important issues around disparities in access to telehealth among patients in medically underserved communities or for those who may not have access to telehealth services.

Our study has some limitations. We did not randomize sites to telehealth versus in-person care, and thus, we are unable to make causal inferences about the impact of telehealth on patient experience. However, we were able to compare patients who received telehealth only versus those who received in-person only, which strengthens the clarity of our observations. We did not constrain eligible visits by type; for example, initial versus established visits. There is also the possibility for recall bias in patients' reported experiences. It is possible that the positive experiences reported by patients are relative to an established provider relationship, although we would have then expected more variation in experiences.

Conclusion

These data from palliative care patients and programs throughout the United States provide initial evidence that telehealth can expand access while not affecting patients' experiences with palliative care with telehealth and offers insight into outpatient palliative care programs' experiences in rapidly adopting telehealth during the COVID-19 pandemic.

Footnotes

Acknowledgment

We thank our research assistant Monica Rico, BS, for her support in this project.

Funding Information

This work was supported by the Centers for Medicare and Medicaid Services under a cooperative agreement referred to as the “Palliative Care Measures Project.” The cooperative agreement number is 1V1CMS331639-01-00.

Author Disclosure Statement

One of the authors, Katherine Ast, is a member of ASCO's end-of-life measures committee.

References

Worster

and Swartz

: Telemedicine and palliative care: An increasing role in supportive oncology. Curr Oncol Rep, 2017; 19:37.

Lupu

: Estimate of current hospice and palliative medicine physician workforce shortage. J Pain Symptom Manage, 2010; 40:899–911.

Hancock

, Preston

, Jones

, and Gadoud

: Telehealth in palliative care is being described but not evaluated: A systematic review. BMC Palliat Care, 2019; 18:114.

Bakitas

, Dionne-Odom

, Ejem

, et al.: Effect of an early palliative care telehealth intervention vs usual care on patients with heart failure: The ENABLE CHF-PC Randomized Clinical Trial. JAMA Intern Med, 2020; 80:1203–1213.

Bonsignore

, Bloom

, Steinhauser

, et al.: Evaluating the feasibility and acceptability of a Telehealth Program in a Rural Palliative Care Population: TapCloud for Palliative Care. J Pain Symptom Manage, 2018; 56:7–14.

Steindal

, Nes

AAG

, Godskesen

, et al.: Patients' experiences of telehealth in palliative home care: Scoping review. J Med Internet Res, 2020; 22:e16218.

Zheng

, Head

, and Schapmire

: A systematic review of telehealth in palliative care: Caregiver outcomes. Telemed J E Health, 2015; 22:288–294.

Head

, Schapmire

, and Zheng

: Telehealth in palliative care: A systematic review of patient-reported outcomes. J Hosp Palliat Nurs, 2017; 19:130–139.

Brook Calton

WPS

, Cohen

, Pantilat

, Rabow

, O'Riordan

, Bischoff

: Patient and caregiver experience with outpatient palliative care telemedicine visits. Palliat Med Rep, 2020; 1:339–346.

10.

Calton

, Abedini

, and Fratkin

: Telemedicine in the time of coronavirus. J Pain Symptom Manage, 60:e12–e14.

11.

Hollander

and Carr

: Virtually perfect? Telemedicine for Covid-19. N Engl J Med, 2020; 382:1679–1681.

12.

Ritchey

, Foy

, McArdel

, and Gruenewald

: Reinventing palliative care delivery in the era of COVID-19: How telemedicine can support end of life care. Am J Hosp Palliat Med, 2020; 37:992–997.

13.

Centers for Medicare and Medicaid: Medicare Telemedicine Health Care Provider Fact Sheet: 2020. https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet (Last accessed October 20, 2021).

14.

Humphreys

, Schoenherr

, Elia

, et al.: Rapid implementation of inpatient telepalliative medicine consultations during COVID-19 pandemic. J Pain Symptom Manage, 2020; 60:e54–e59.

15.

Wakam

, Montgomery

, Biesterveld

, and Brown

: Not dying alone—Modern compassionate care in the Covid-19 pandemic. N Engl J Med, 2020; 382:e88.

16.

Lally

, Kematick

, Gorman

, and Tulsky

: Rapid conversion of a palliative care outpatient clinic to telehealth. JCO Oncol Pract 17:e62–e67.

17.

Parast

, Elliott

, Hambarsoomian

, et al.: Effects of survey mode on consumer assessment of healthcare providers and systems (CAHPS) hospice survey scores. J Am Geriatr Soc, 2018; 66:546–552.

18.

Dillman

, Smyth

, and Christian

: Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley & Sons, 2014.

19.

Safran

, Karp

, Coltin

, et al.: Measuring patients' experiences with individual primary care physicians. Results of a statewide demonstration project. J Gen Intern Med, 2006; 21:13–21.

20.

Mack

, Block

, Nilsson

, et al.: Measuring therapeutic alliance between oncologists and patients with advanced cancer: The Human Connection Scale. Cancer, 2009; 115:3302–3311.

21.

Gramling

, Stanek

, Ladwig

, et al.: Feeling heard and understood: A patient-reported quality measure for the inpatient palliative care setting. J Pain Symptom Manage, 2016; 51:150–154.

22.

Ingersoll

, Saeed

, Ladwig

, et al.: Feeling heard and understood in the hospital environment: Benchmarking communication quality among patients with advanced cancer before and after palliative care consultation. J Pain Symptom Manage, 2018; 56:239–244.

23.

Cassady

, Starfield

, Hurtado

, et al.: Measuring consumer experiences with primary care. Pediatrics, 2000; 105(4 Pt 2):998–1003.

24.

Safran

, Kosinski

, Tarlov

, et al.: The Primary Care Assessment Survey: Tests of data quality and measurement performance. Med Care, 1998; 36:728–739.

25.

CAHPS Clinician & Group Survey: Agency for Healthcare Research and Quality. 2021. https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html (Last accessed October 20, 2021).

26.

Johnson

and Vindrola-Padros

: Rapid qualitative research methods during complex health emergencies: A systematic review of the literature. Soc Sci Med, 2017; 189:63–75.

27.

Taylor

, Henshall

, Kenyon

, et al.: Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open, 2018; 8:e019993.

28.

Gale

, Wu

, Erhardt

, et al.: Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci, 2019; 14:11.

29.

Carrillo de Albornoz

, Sia

, and Harris

: The effectiveness of teleconsultations in primary care: Systematic review. Fam Pract, 2022; 39:168–182.

30.

Kruse

, Krowski

, Rodriguez

, et al.: Telehealth and patient satisfaction: A systematic review and narrative analysis. BMJ Open, 2017; 7:e016242.

31.

Almathami

HKY

, Win

, and Vlahu-Gjorgievska

: Barriers and facilitators that influence telemedicine-based, real-time, online consultation at patients' homes: Systematic literature review. J Med Internet Res, 2020; 22:e16407.

32.

Orlando

, Beard

, and Kumar

: Systematic review of patient and caregivers' satisfaction with telehealth videoconferencing as a mode of service delivery in managing patients' health. PLoS One, 2019; 14:e0221848.

33.

Lupu

, Quigley

, Mehfoud

, and Salsberg

: The growing demand for hospice and palliative medicine physicians: Will the supply keep up?. J Pain Symptom Manage, 2018; 55:1216–1223.

34.

Eberly

, Kallan

, Julien

, et al.: Patient characteristics associated with telemedicine access for primary and specialty ambulatory care during the COVID-19 pandemic. JAMA Netw Open, 2020; 3:e2031640.